-- Logs begin at Mon 2022-07-18 13:36:46 UTC, end at Mon 2022-07-18 14:41:19 UTC. -- Jul 18 14:04:04.969767 dv-cinder-797608-58-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 18 14:04:04.978775 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 18 14:04:04 2022] *** Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: nodename: dv-cinder-797608-58-iscsi Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: machine: x86_64 Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: clock source: unix Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: pcre jit disabled Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: detected number of CPU cores: 4 Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: current working directory: / Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: detected binary path: /usr/bin/uwsgi-core Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: your processes number limit is 47680 Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: your memory page size is 4096 bytes Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: detected max file descriptor number: 2048 Jul 18 14:04:04.980946 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: lock engine: pthread robust mutexes Jul 18 14:04:04.981705 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: thunder lock: enabled Jul 18 14:04:04.981705 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 18 14:04:04.981705 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 18 14:04:05.020950 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: Python main interpreter initialized at 0x55f1223ab9c0 Jul 18 14:04:05.020950 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: python threads support enabled Jul 18 14:04:05.020950 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: your server socket listen backlog is limited to 100 connections Jul 18 14:04:05.020950 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: your mercy for graceful operations on workers is 90 seconds Jul 18 14:04:05.021325 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: mapped 403077 bytes (393 KB) for 2 cores Jul 18 14:04:05.021402 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: *** Operational MODE: preforking *** Jul 18 14:04:05.021687 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: *** uWSGI is running in multiple interpreter mode *** Jul 18 14:04:05.021687 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: spawned uWSGI master process (pid: 80203) Jul 18 14:04:05.021929 dv-cinder-797608-58-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 18 14:04:05.022746 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: spawned uWSGI worker 1 (pid: 80213, cores: 1) Jul 18 14:04:05.025688 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: spawned uWSGI worker 2 (pid: 80214, cores: 1) Jul 18 14:04:05.025688 dv-cinder-797608-58-iscsi devstack@keystone.service[80203]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 18 14:04:10.565402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 14:04:10.565402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: from cryptography.utils import int_from_bytes Jul 18 14:04:10.566277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 14:04:10.566277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: from cryptography.utils import int_from_bytes Jul 18 14:04:10.566277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 14:04:10.566277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: from cryptography.utils import int_from_bytes Jul 18 14:04:10.566769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 18 14:04:10.566769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: from cryptography.utils import int_from_bytes Jul 18 14:04:10.614600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 14:04:10.614793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 14:04:10.614970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] command line args: [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 14:04:10.615116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 18 14:04:10.615206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 14:04:10.615368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 18 14:04:10.615495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ================================================================================ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 14:04:10.615581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] command line args: [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 18 14:04:10.615649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] admin_token = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.615789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 18 14:04:10.615869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] config_dir = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.616005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ================================================================================ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 18 14:04:10.616147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.616215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] config_source = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.616279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] admin_token = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.616352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] debug = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.616493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_dir = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.616618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.617193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.617193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] config_source = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.617324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.617393 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] debug = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.617458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.617521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.617935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_config_append = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_dir = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_file = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_config_append = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.618952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_dir = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619741 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.619970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.620897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.621972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.622179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.622326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.622406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.622755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.622855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.622938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] publish_errors = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.623054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.623168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.71/identity {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.623253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.623333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] publish_errors = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.623416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.623599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.623818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.623904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.624054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.624200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.624327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] standard_threads = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.624424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.624555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.624722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.624902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] standard_threads = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_journal = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_json = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_stderr = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] use_syslog = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_journal = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.625993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_json = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.626179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.626315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_stderr = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.626391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.626504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] use_syslog = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.626585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.626664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 18 14:04:10.626735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.626809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.626885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.626956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.627024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.627100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.627100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.627267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.627343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.627413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.627999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.628955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629257 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.629923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.630977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.631087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.631205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.631323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.631442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.631554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.631664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.631779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.631897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632355 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632813 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.632925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.633043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.633225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.633405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.633549 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.633704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.633837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.633952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.634063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.634175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.634291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.634389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.634491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.634592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.634687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.634872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.635042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.635149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.635245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.635347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.635444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.635538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.635768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.635938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.636100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.636268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.636450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.636609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.636609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.636806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.636806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.636959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.637018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.637194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.637319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.637463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.637636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.637786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.637902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.638935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.639089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.639214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.639326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.639471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.639590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.639704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.639817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.639928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.640046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.640209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.640336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.640449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.640558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.640671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.640781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.640891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.641011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.641134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.641259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.641384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.641535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.641712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.641829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.641955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.642068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.642178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.642297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.642409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.642582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.642702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.642826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.642939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.643082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.643794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.643912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.644951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.645123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.645206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.645387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.645469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.645698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.645848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.645956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.646937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.647087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.647180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.647330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.647413 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.647570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.647653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.647836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.648018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.648103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.648178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.648299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.648391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.648550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.648643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.648795 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.649983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.650068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.650220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.650345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.650688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.650784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.650861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.650950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.651012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.651139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.651280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.651922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.external = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.external = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.652932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.653076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.653174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.password = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.653376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] auth.token = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.653477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.653616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.653724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.653876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.password = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.653971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.654113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] auth.token = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.654212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.654353 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.654438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.654596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.654749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.654835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.655036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.655121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.655304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.655397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.655559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.655643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.655792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.655876 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.656031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.656124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.656266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.656408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.656534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.656658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.656795 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.656917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.657040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.657162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.657292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.657433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.657533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.657722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.657827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.657968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.658087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.658228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.658329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.658522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.658605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.658775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.658855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.659044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.659124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.659294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.659383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.659556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.659644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.659798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.659879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.660042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.660124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.660282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.660361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.660533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.660614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.660767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.660848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.661004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.661083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.661248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.661328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.661491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.661571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.661762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.661845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.662969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.663996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.664154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.664242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.664470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.664634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.664715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.664887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.664976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.665122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.665201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.665359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.665439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.665590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.665711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.665854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.666002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.666082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.666155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.666332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.666422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.666590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.666673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.666842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.667089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.667285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.667652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.667733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.667806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.667879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.667952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.668938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.669018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.669212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.669299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.669434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.669525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.669690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.669807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.669951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.670057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.670207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.670306 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.670453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.670541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.670706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.670793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.670959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.671987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.672141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.672228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.672380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.672475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.672678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.672761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.672834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.672905 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.673087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.673174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.673337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.673415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.673576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.673680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.673835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.673932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.674072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.674152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.674324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.674403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.674570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.674673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.674809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.674896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.675048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.675125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.675285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.675372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.675526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.675603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.675772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.675851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.676997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.677076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.677233 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.677312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.677464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.677544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.677724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.677809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.677964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.678043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.678198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.678277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.678430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.678508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.678680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.678758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.678929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.679971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.680145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.680223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.680385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.680465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.680624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.680702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.680861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.680939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.681102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.681182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.681351 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.681428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.681595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.681707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.681861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.681941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.682105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.682187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.682341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.682418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.682660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.682879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.682944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.683975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.684062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.684216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.684293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.684458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.684536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.684710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.684797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.684960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.685037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.685197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.685274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.685450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.685537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.685725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.685809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.685964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.686043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.686213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.686374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.686450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.686521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.686685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.686764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.686925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] role.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] role.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.687986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.688157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.688237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.688408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.688494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.688656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.688735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.688910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.688987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.689160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.689240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.689411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.689488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.689647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.689752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.689917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.689998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.690168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.690247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.690418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.690496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.690669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.690751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.691104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.691184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.691344 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.691421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.691595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.691675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.691845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.691923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.692094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.692172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.692346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.692427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.692584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.692663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.692842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.692920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.693102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.693181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.693499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.693592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.693592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.693860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.693860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.694896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.695053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.695145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.695289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.695370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.695524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.695605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.695764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.695854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.696007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.696088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.696257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.696338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.696496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.696585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.696752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.696833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.697000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.697080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.697237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.697326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.697484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.697563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.697756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.697842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.698006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.698204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.698294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.698816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.698816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.698966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.698966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.699953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.700161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.700242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.700421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.700515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.700669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.700758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.700920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.701001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.701159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.701239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.701402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.701483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.701641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.701736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.701906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.702959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.703131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.703219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.703373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.703453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.703624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.703704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.703861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.703941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704666 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 18 14:04:10.704994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80213) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 14:04:10.705075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80214) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 18 14:04:10.736097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.736327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.736420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.736554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.736646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.736816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.745457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.745911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.773888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.774077 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.774176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.774252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.917862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.918590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.967287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.967534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.968760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:10.969032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.032627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.032825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.032896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.033905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.058916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059666 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059750 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.059973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.060097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.060167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.060244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.060441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.071496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.071989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.084346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.084346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.084754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.084829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.084906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.084968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.085037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.085197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.085323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.085393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.085456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.085607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80214) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.085777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.085863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80213) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 18 14:04:11.103046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.103236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 14:04:11.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.103548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.103625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 14:04:11.103783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 14:04:11.103909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.104100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.104224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 18 14:04:11.104308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.104385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.104793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.104918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.105086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.105196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.105420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.116088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.116372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.116497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.116655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.116770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.116917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.117043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.117221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.125360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.125867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.128650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.128934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.129094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.129191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.129454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.129963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.130951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.131156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.131283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.131405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.131486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.131714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.131962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.157883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.158445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.159497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.159775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.160074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.160361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.165020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.165298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.165577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.165749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.165917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.166200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.173143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.173374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.173513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.173816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.178009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.178297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.178421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.178558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.178690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.178871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.178998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.179129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.179211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.179423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.179551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.179672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.179765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.179955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.180083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.180427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.199037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.199316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.199488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.199730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.203858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.204280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.207570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.207889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.207985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.208303 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.212908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.213365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.216621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.217099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.220402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.220701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.220885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.220972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.221189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.221291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.221533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.221627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.221756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.221833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.222070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.222357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.240882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.241281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.241524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.241703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.241810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.241900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.242156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.242428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.250197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.250744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.251956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.252224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.252478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.252753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.255299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.255588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.255800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.255895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.256105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.256212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.256392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.256478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.256669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.256967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.269968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.270253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.270531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.270788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.270916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.271047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.271139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.271357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.271485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.271600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.271704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.271998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.272152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.272464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.291249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.292381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.294756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.295747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.298071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.299068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.299667 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.300663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.301293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.301575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.301882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.302326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.302607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.302887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.309360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.310594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.311005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.312214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.314384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.314670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.314950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.315223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.315488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.315783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.316063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.316339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.324707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.325128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.325253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.325928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.326228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.326511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.333121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80214) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.333403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.333689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.333977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.334258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.334347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80213) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 18 14:04:11.334549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.334649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.334818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.334903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.335084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.335168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.335358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.335443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.335658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.335757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.335895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.335974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.336170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.336255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.336432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.336525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.336693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80214) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.336778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.336968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.337244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.337511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.337797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80213) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 18 14:04:11.375171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f1223ab9c0 pid: 80214 (default app) Jul 18 14:04:11.376137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f1223ab9c0 pid: 80213 (default app) Jul 18 14:04:11.378733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d294f-8b8f-4bc6-b6f0-8bef5da38562 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:11.379031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d294f-8b8f-4bc6-b6f0-8bef5da38562 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:11.379310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d294f-8b8f-4bc6-b6f0-8bef5da38562 None None] PATH_INFO: `/v3` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:11.383599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1/1] 10.222.0.71 () {50 vars in 791 bytes} [Mon Jul 18 14:04:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:19.192078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272579fe-8ce1-4ff5-bde3-282e0a2f5536 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:19.192702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272579fe-8ce1-4ff5-bde3-282e0a2f5536 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:19.192702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272579fe-8ce1-4ff5-bde3-282e0a2f5536 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:19.197261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1/2] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:04:19.203864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:19.204431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:19.204431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:19.273675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: INFO dbcounter [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] Registered counter for database keystone Jul 18 14:04:19.296149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG oslo_db.sqlalchemy.engines [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80214) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 14:04:19.303886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writer thread running {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 14:04:19.489413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG passlib.handlers.bcrypt [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80214) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 14:04:19.489781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG passlib.handlers.bcrypt [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80214) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 14:04:19.546383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:19.547964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:19.678140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 2/3] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 14:04:19.835543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272579fe-8ce1-4ff5-bde3-282e0a2f5536 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:19.906542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: INFO dbcounter [None req-272579fe-8ce1-4ff5-bde3-282e0a2f5536 None None] Registered counter for database keystone Jul 18 14:04:19.926930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG oslo_db.sqlalchemy.engines [None req-272579fe-8ce1-4ff5-bde3-282e0a2f5536 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80213) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 18 14:04:19.935139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writer thread running {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 18 14:04:20.057111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272579fe-8ce1-4ff5-bde3-282e0a2f5536 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:20.062782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c51cdf-5137-4bb8-b96e-57b7b211a9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:20.063465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c51cdf-5137-4bb8-b96e-57b7b211a9bb admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:20.063753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c51cdf-5137-4bb8-b96e-57b7b211a9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:20.064028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c51cdf-5137-4bb8-b96e-57b7b211a9bb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:20.073489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c51cdf-5137-4bb8-b96e-57b7b211a9bb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:20.127238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b9c51cdf-5137-4bb8-b96e-57b7b211a9bb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 14:04:20.127457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c51cdf-5137-4bb8-b96e-57b7b211a9bb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:20.138845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 2/4] 10.222.0.71 () {58 vars in 1324 bytes} [Mon Jul 18 14:04:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:04:20.145223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:20.161240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5c27a-373c-4df5-90a8-7ef64cec46a7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:20.166923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ed3c9-fd17-42b6-964c-8cd1963afa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:20.168165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed3c9-fd17-42b6-964c-8cd1963afa4d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:20.168462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed3c9-fd17-42b6-964c-8cd1963afa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:20.168735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed3c9-fd17-42b6-964c-8cd1963afa4d admin admin] PATH_INFO: `/v3/projects/c1db40e87aaf497fb0528cd53913ada9` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:20.173186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ed3c9-fd17-42b6-964c-8cd1963afa4d admin admin] RBAC: Authorizing `identity:get_project(project_id=c1db40e87aaf497fb0528cd53913ada9)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:20.223349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb7ed3c9-fd17-42b6-964c-8cd1963afa4d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 18 14:04:20.224056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ed3c9-fd17-42b6-964c-8cd1963afa4d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:20.228492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 3/5] 10.222.0.71 () {56 vars in 1207 bytes} [Mon Jul 18 14:04:20 2022] GET /identity/v3/projects/c1db40e87aaf497fb0528cd53913ada9 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:22.302837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e3e89-e3ae-4ed8-ace0-f841d0d43213 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:22.303423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e3e89-e3ae-4ed8-ace0-f841d0d43213 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:22.303423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e3e89-e3ae-4ed8-ace0-f841d0d43213 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:22.307669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 3/6] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:22.314102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532539d3-64e1-445d-9ff8-e8b21953edb3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:22.314650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532539d3-64e1-445d-9ff8-e8b21953edb3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:22.314650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532539d3-64e1-445d-9ff8-e8b21953edb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:22.334752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-532539d3-64e1-445d-9ff8-e8b21953edb3 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:22.336339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-532539d3-64e1-445d-9ff8-e8b21953edb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:22.345100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 4/7] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 18 14:04:22.368355 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e3e89-e3ae-4ed8-ace0-f841d0d43213 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:22.383025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e3e89-e3ae-4ed8-ace0-f841d0d43213 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:22.387685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5efc0-ca3c-4f36-b6d9-c9de550ed64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:22.388338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5efc0-ca3c-4f36-b6d9-c9de550ed64f admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:22.388628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5efc0-ca3c-4f36-b6d9-c9de550ed64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:22.388912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5efc0-ca3c-4f36-b6d9-c9de550ed64f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:22.397558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5efc0-ca3c-4f36-b6d9-c9de550ed64f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:22.399402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5efc0-ca3c-4f36-b6d9-c9de550ed64f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:22.409428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 4/8] 10.222.0.71 () {58 vars in 1324 bytes} [Mon Jul 18 14:04:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:04:22.416606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532539d3-64e1-445d-9ff8-e8b21953edb3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:22.431937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532539d3-64e1-445d-9ff8-e8b21953edb3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:22.437356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dba6ca-6e92-4fc0-8d7f-9ec693df3b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:22.438296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dba6ca-6e92-4fc0-8d7f-9ec693df3b98 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:22.438584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dba6ca-6e92-4fc0-8d7f-9ec693df3b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:22.438836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dba6ca-6e92-4fc0-8d7f-9ec693df3b98 admin admin] PATH_INFO: `/v3/users/86f9463b664941f0b5c63b336b679b58` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:22.440323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dba6ca-6e92-4fc0-8d7f-9ec693df3b98 admin admin] RBAC: Authorizing `identity:get_user(user_id=86f9463b664941f0b5c63b336b679b58)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:22.442246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dba6ca-6e92-4fc0-8d7f-9ec693df3b98 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:22.443933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 5/9] 10.222.0.71 () {56 vars in 1198 bytes} [Mon Jul 18 14:04:22 2022] GET /identity/v3/users/86f9463b664941f0b5c63b336b679b58 => generated 247 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:24.560580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00db8bf-4af1-4cad-9d83-aed5da3d4b26 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:24.561192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00db8bf-4af1-4cad-9d83-aed5da3d4b26 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:24.561801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00db8bf-4af1-4cad-9d83-aed5da3d4b26 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:24.563386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 5/10] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:24.569583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd486773-ef86-4f51-a71c-10a69c290912 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:24.570402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd486773-ef86-4f51-a71c-10a69c290912 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:24.570402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd486773-ef86-4f51-a71c-10a69c290912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:24.589241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bd486773-ef86-4f51-a71c-10a69c290912 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:24.590554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bd486773-ef86-4f51-a71c-10a69c290912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:24.649878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e314-c75b-4a75-b818-a34c6d056988 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:24.650152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e314-c75b-4a75-b818-a34c6d056988 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:24.650635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73e314-c75b-4a75-b818-a34c6d056988 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:24.652115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 6/11] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:04:24.658440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:24.658820 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:24.659109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:24.701519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 6/12] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:24.701795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG passlib.handlers.bcrypt [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80213) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 18 14:04:24.702158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG passlib.handlers.bcrypt [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80213) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 18 14:04:24.725519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd486773-ef86-4f51-a71c-10a69c290912 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:24.748197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:24.749950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:24.754814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:24.851172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd486773-ef86-4f51-a71c-10a69c290912 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:24.879303 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 7/13] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:24.889685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21966e-bdf5-444a-aede-c991698bf8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:24.890508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21966e-bdf5-444a-aede-c991698bf8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:24.890801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21966e-bdf5-444a-aede-c991698bf8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:24.891074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21966e-bdf5-444a-aede-c991698bf8c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:24.903841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb21966e-bdf5-444a-aede-c991698bf8c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:24.904437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-eb21966e-bdf5-444a-aede-c991698bf8c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:24.915350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:24.916520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:24.918577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 7/14] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:04:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:24.922746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21966e-bdf5-444a-aede-c991698bf8c8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:25.021899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b5eb7-5df2-4091-a63b-8d824aeea7d9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:25.022106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:25.024367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:25.043731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb21966e-bdf5-444a-aede-c991698bf8c8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:25.064271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6103fe0-d8f0-4568-80e7-3283e86a0d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:25.065207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6103fe0-d8f0-4568-80e7-3283e86a0d4c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:25.065502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6103fe0-d8f0-4568-80e7-3283e86a0d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:25.065820 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6103fe0-d8f0-4568-80e7-3283e86a0d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:25.084589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:25.085271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:25.085543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:25.085858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:25.086611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:25.086923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:25.088366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:25.102913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 8/15] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:25.108901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:25.146177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6103fe0-d8f0-4568-80e7-3283e86a0d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:25.147731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6103fe0-d8f0-4568-80e7-3283e86a0d4c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:25.252217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718be6c-cdf5-46bb-b651-63373819365d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:25.256017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 8/16] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:25.261839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6103fe0-d8f0-4568-80e7-3283e86a0d4c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:25.266101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:25.293634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7fdbb-6dc9-4f79-94e0-e20d086bd7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:25.294556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7fdbb-6dc9-4f79-94e0-e20d086bd7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:25.294842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7fdbb-6dc9-4f79-94e0-e20d086bd7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:25.295125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c7fdbb-6dc9-4f79-94e0-e20d086bd7f2 None admin] PATH_INFO: `/v3/users/86f9463b664941f0b5c63b336b679b58` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:25.296395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7fdbb-6dc9-4f79-94e0-e20d086bd7f2 None admin] RBAC: Authorizing `identity:get_user(user_id=86f9463b664941f0b5c63b336b679b58)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:25.297925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c7fdbb-6dc9-4f79-94e0-e20d086bd7f2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:25.300225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 9/17] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:25 2022] GET /identity/v3/users/86f9463b664941f0b5c63b336b679b58 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:25.305578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7fdbb-6dc9-4f79-94e0-e20d086bd7f2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:25.321507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:25.379463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6103fe0-d8f0-4568-80e7-3283e86a0d4c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:25.415474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:25.416294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:25.416558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:25.416830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:25.418136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c7fdbb-6dc9-4f79-94e0-e20d086bd7f2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:25.437887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:25.439390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:25.455968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebcbf2-3879-4303-8533-a24ccbc07b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:25.456774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcbf2-3879-4303-8533-a24ccbc07b75 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:25.457056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcbf2-3879-4303-8533-a24ccbc07b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:25.457450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebcbf2-3879-4303-8533-a24ccbc07b75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:25.458823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebcbf2-3879-4303-8533-a24ccbc07b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:25.460340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebcbf2-3879-4303-8533-a24ccbc07b75 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:04:25.461395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:25.462475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:25.463686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:04:25.463686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:04:25.463686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:04:25.463686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:25.464120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 10/18] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:25.464232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 9/19] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:25.469254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebcbf2-3879-4303-8533-a24ccbc07b75 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:25.469574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:25.531070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:25.559030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:25.580302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb8eb9-6d5f-4656-8037-07b59a167adf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:25.583927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebcbf2-3879-4303-8533-a24ccbc07b75 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:25.616656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36051e65-3e72-4d46-bc06-222203ad79f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:25.617532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051e65-3e72-4d46-bc06-222203ad79f5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:25.618145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051e65-3e72-4d46-bc06-222203ad79f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:25.618657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36051e65-3e72-4d46-bc06-222203ad79f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:25.619594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36051e65-3e72-4d46-bc06-222203ad79f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:25.620778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36051e65-3e72-4d46-bc06-222203ad79f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:25.620874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f9e248-86dc-4a57-9372-3854163d0b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:25.621822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e248-86dc-4a57-9372-3854163d0b8c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:25.622202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e248-86dc-4a57-9372-3854163d0b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:25.622542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f9e248-86dc-4a57-9372-3854163d0b8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:25.623211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36051e65-3e72-4d46-bc06-222203ad79f5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:25.623626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9e248-86dc-4a57-9372-3854163d0b8c None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.domain.id=default, user.id=86f9463b664941f0b5c63b336b679b58) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:25.624005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9e248-86dc-4a57-9372-3854163d0b8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:25.625940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f9e248-86dc-4a57-9372-3854163d0b8c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:25.634223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 11/20] 10.222.0.71 () {56 vars in 1322 bytes} [Mon Jul 18 14:04:25 2022] GET /identity/v3/role_assignments?user.id=86f9463b664941f0b5c63b336b679b58&scope.domain.id=default&role.id=5c672a1b2979444383c587a215adeb35 => generated 233 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:25.646861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 10/21] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:28.163649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0526e-823c-4b5f-8df4-1cc9cc1ac01c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.164278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0526e-823c-4b5f-8df4-1cc9cc1ac01c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.164278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0526e-823c-4b5f-8df4-1cc9cc1ac01c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.165824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 12/22] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:28.171493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a986df-f241-4fba-ae65-1ea1d808b474 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.172010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a986df-f241-4fba-ae65-1ea1d808b474 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.172098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a986df-f241-4fba-ae65-1ea1d808b474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.195370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f3a986df-f241-4fba-ae65-1ea1d808b474 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:28.196610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f3a986df-f241-4fba-ae65-1ea1d808b474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:28.292529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 11/23] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:28.327662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906101-759d-4698-81e7-0674a651628b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.327995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906101-759d-4698-81e7-0674a651628b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.328230 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20906101-759d-4698-81e7-0674a651628b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.329629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 13/24] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:28.330676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a986df-f241-4fba-ae65-1ea1d808b474 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:28.344740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b67492-e8b0-432b-b393-5760365bc104 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.345060 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b67492-e8b0-432b-b393-5760365bc104 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.345302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b67492-e8b0-432b-b393-5760365bc104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.381210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-69b67492-e8b0-432b-b393-5760365bc104 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:28.382464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-69b67492-e8b0-432b-b393-5760365bc104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:28.463143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:28.473535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 14/25] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 18 14:04:28.525884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b67492-e8b0-432b-b393-5760365bc104 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:28.569017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a986df-f241-4fba-ae65-1ea1d808b474 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:28.651638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:28.662253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb827f9f-1f36-4217-ba1f-b336b2ffeb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:28.662873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827f9f-1f36-4217-ba1f-b336b2ffeb77 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.663160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827f9f-1f36-4217-ba1f-b336b2ffeb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.663448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb827f9f-1f36-4217-ba1f-b336b2ffeb77 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.664227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb827f9f-1f36-4217-ba1f-b336b2ffeb77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:28.673745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb827f9f-1f36-4217-ba1f-b336b2ffeb77 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:28.684311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b67492-e8b0-432b-b393-5760365bc104 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:28.738506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ea0af-3792-4f0f-85c9-5e55ca951afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:28.739685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ea0af-3792-4f0f-85c9-5e55ca951afa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.740176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ea0af-3792-4f0f-85c9-5e55ca951afa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.740636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ea0af-3792-4f0f-85c9-5e55ca951afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.754662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ea0af-3792-4f0f-85c9-5e55ca951afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:28.756644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ea0af-3792-4f0f-85c9-5e55ca951afa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-823ea0af-3792-4f0f-85c9-5e55ca951afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:04:28.778467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:28.779342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:28.780122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:04:28.780122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:04:28.780122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:04:28.780122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:28.782225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 15/26] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:28.783821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c4ebd-a30d-4c40-82d3-9c86c8b12580 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.784404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c4ebd-a30d-4c40-82d3-9c86c8b12580 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.784887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c4ebd-a30d-4c40-82d3-9c86c8b12580 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.786507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 16/27] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:28.787985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb249b-f145-4c8d-8812-0082c80f02d8 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.788475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb249b-f145-4c8d-8812-0082c80f02d8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.788937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb249b-f145-4c8d-8812-0082c80f02d8 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.791052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 17/28] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:28.792814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cca08-42ca-4af1-a663-7f1eba540919 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.793352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cca08-42ca-4af1-a663-7f1eba540919 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.794035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5cca08-42ca-4af1-a663-7f1eba540919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.812986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5c5cca08-42ca-4af1-a663-7f1eba540919 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:28.814631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5c5cca08-42ca-4af1-a663-7f1eba540919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:28.837217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 12/29] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 18 14:04:28 2022] POST /identity/v3/roles => generated 219 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:28.838319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb827f9f-1f36-4217-ba1f-b336b2ffeb77 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:28.906334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 18/30] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:28.908247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aac96-c3c2-4d92-8cb5-17d936fb1736 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:28.908774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aac96-c3c2-4d92-8cb5-17d936fb1736 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:28.909273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604aac96-c3c2-4d92-8cb5-17d936fb1736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:28.931477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-604aac96-c3c2-4d92-8cb5-17d936fb1736 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:28.933458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-604aac96-c3c2-4d92-8cb5-17d936fb1736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:29.018806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb827f9f-1f36-4217-ba1f-b336b2ffeb77 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:29.038039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:29.107302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:29.109207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:29.110829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:29.114311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:29.115048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:29.116756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:29.119151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:29.125333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:29.160156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 19/31] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:29.161217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604aac96-c3c2-4d92-8cb5-17d936fb1736 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:29.162947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 13/32] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:29.169218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:29.354409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d0e7a-b47b-4e68-bff9-509523170540 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:29.393582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbec72-4f89-451b-85bd-e489d0e4b9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:29.394503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbec72-4f89-451b-85bd-e489d0e4b9ea None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:29.394892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbec72-4f89-451b-85bd-e489d0e4b9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:29.395277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbec72-4f89-451b-85bd-e489d0e4b9ea None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:29.396426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbec72-4f89-451b-85bd-e489d0e4b9ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:29.398158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbec72-4f89-451b-85bd-e489d0e4b9ea None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:29.420776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604aac96-c3c2-4d92-8cb5-17d936fb1736 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:29.441220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 14/33] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:04:29 2022] POST /identity/v3/projects => generated 301 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:04:29.443220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbec72-4f89-451b-85bd-e489d0e4b9ea None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:29.488451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:29.499013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67c4a2-bcaf-4415-b2ee-3f37e98b13c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:29.501116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c4a2-bcaf-4415-b2ee-3f37e98b13c9 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:29.501629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c4a2-bcaf-4415-b2ee-3f37e98b13c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:29.502088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c4a2-bcaf-4415-b2ee-3f37e98b13c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:29.503460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67c4a2-bcaf-4415-b2ee-3f37e98b13c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:29.516591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67c4a2-bcaf-4415-b2ee-3f37e98b13c9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:29.669768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbec72-4f89-451b-85bd-e489d0e4b9ea None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:29.741941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d676f02-43fd-4fdd-a9b1-f1da6e5aab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:29.744208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d676f02-43fd-4fdd-a9b1-f1da6e5aab9d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:29.745025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d676f02-43fd-4fdd-a9b1-f1da6e5aab9d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:29.745847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d676f02-43fd-4fdd-a9b1-f1da6e5aab9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:29.769081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d676f02-43fd-4fdd-a9b1-f1da6e5aab9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1d676f02-43fd-4fdd-a9b1-f1da6e5aab9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:29.786772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:29.788482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:29.790016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:29.794188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 15/34] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:04:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:29.799583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 20/35] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 18 14:04:29 2022] POST /identity/v3/roles => generated 225 bytes in 638 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:29.806832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d676f02-43fd-4fdd-a9b1-f1da6e5aab9d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:29.921917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae8ae2-58ad-43e1-9dd9-bf6e16309d41 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:29.922168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae8ae2-58ad-43e1-9dd9-bf6e16309d41 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:29.922498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae8ae2-58ad-43e1-9dd9-bf6e16309d41 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:29.923795 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 21/36] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:04:29.930789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc184678-c9a6-4c99-b306-d07062856348 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:29.931191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc184678-c9a6-4c99-b306-d07062856348 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:29.931361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc184678-c9a6-4c99-b306-d07062856348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:29.951571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fc184678-c9a6-4c99-b306-d07062856348 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:29.953417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fc184678-c9a6-4c99-b306-d07062856348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:29.990196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:30.016326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d676f02-43fd-4fdd-a9b1-f1da6e5aab9d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:30.043296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:30.079245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:30.080799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.081313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.081830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.083284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:30.083849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:30.085115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 22/37] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:30.086527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c379db-4b03-4fb2-8158-f2500b85aa43 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.086828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c379db-4b03-4fb2-8158-f2500b85aa43 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.087109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c379db-4b03-4fb2-8158-f2500b85aa43 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.088567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:30.088696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 23/38] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:30.095923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea5b8a-f0e9-45d4-a334-31c4b1ae189b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.096219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea5b8a-f0e9-45d4-a334-31c4b1ae189b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.096501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cea5b8a-f0e9-45d4-a334-31c4b1ae189b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.118524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 16/39] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:30.120257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9cea5b8a-f0e9-45d4-a334-31c4b1ae189b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:30.120450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:30.121622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9cea5b8a-f0e9-45d4-a334-31c4b1ae189b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:30.201104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 24/40] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:30.201925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea5b8a-f0e9-45d4-a334-31c4b1ae189b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:30.269369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776a6e3-4a8d-43fb-be87-2be852deb58c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:30.309720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3b274-1fed-46f7-8ef8-335ec7bf25c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:30.310436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3b274-1fed-46f7-8ef8-335ec7bf25c9 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.310720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3b274-1fed-46f7-8ef8-335ec7bf25c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.311009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3b274-1fed-46f7-8ef8-335ec7bf25c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.311780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3b274-1fed-46f7-8ef8-335ec7bf25c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:30.313200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3b274-1fed-46f7-8ef8-335ec7bf25c9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:30.316198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cea5b8a-f0e9-45d4-a334-31c4b1ae189b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:30.323258 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:30.327314 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:30.329543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 17/41] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 18 14:04:30 2022] POST /identity/v3/roles => generated 223 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:04:30.330829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3b274-1fed-46f7-8ef8-335ec7bf25c9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:30.364127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93928a-8825-45f3-aeaa-10be32d89418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:30.365053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93928a-8825-45f3-aeaa-10be32d89418 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.365345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93928a-8825-45f3-aeaa-10be32d89418 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.365629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93928a-8825-45f3-aeaa-10be32d89418 None admin] PATH_INFO: `/v3/users/86f9463b664941f0b5c63b336b679b58` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.367055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93928a-8825-45f3-aeaa-10be32d89418 None admin] RBAC: Authorizing `identity:get_user(user_id=86f9463b664941f0b5c63b336b679b58)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:30.369055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93928a-8825-45f3-aeaa-10be32d89418 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:30.371477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 25/42] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:30 2022] GET /identity/v3/users/86f9463b664941f0b5c63b336b679b58 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:30.377118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93928a-8825-45f3-aeaa-10be32d89418 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:30.484611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3b274-1fed-46f7-8ef8-335ec7bf25c9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:30.531343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35952c33-d419-4a3c-b816-2284b5838f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:30.532099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35952c33-d419-4a3c-b816-2284b5838f25 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.532384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35952c33-d419-4a3c-b816-2284b5838f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.532658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35952c33-d419-4a3c-b816-2284b5838f25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.533973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35952c33-d419-4a3c-b816-2284b5838f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:30.535551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35952c33-d419-4a3c-b816-2284b5838f25 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:30.537134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 18/43] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:30.545624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35952c33-d419-4a3c-b816-2284b5838f25 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:30.613484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93928a-8825-45f3-aeaa-10be32d89418 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:30.623844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:30.651327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a1ba0-e140-4afb-b418-1c2067412bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:30.652700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a1ba0-e140-4afb-b418-1c2067412bec None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.653159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a1ba0-e140-4afb-b418-1c2067412bec None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.653159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205a1ba0-e140-4afb-b418-1c2067412bec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.654515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a1ba0-e140-4afb-b418-1c2067412bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:30.656016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205a1ba0-e140-4afb-b418-1c2067412bec None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:30.657878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 26/44] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:30.663347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a1ba0-e140-4afb-b418-1c2067412bec None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:30.673374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35952c33-d419-4a3c-b816-2284b5838f25 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:30.711641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba6e4f-3cb1-4104-a90b-dffdf64cf1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:30.712360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba6e4f-3cb1-4104-a90b-dffdf64cf1b2 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.712655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba6e4f-3cb1-4104-a90b-dffdf64cf1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.712933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba6e4f-3cb1-4104-a90b-dffdf64cf1b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.713691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba6e4f-3cb1-4104-a90b-dffdf64cf1b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:30.714034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:30.715205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba6e4f-3cb1-4104-a90b-dffdf64cf1b2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:30.744514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 19/45] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:04:30 2022] POST /identity/v3/projects => generated 312 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:04:30.776387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205a1ba0-e140-4afb-b418-1c2067412bec None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:30.817780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02cbe4-c07e-4ebb-90b7-2cb0f05de718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:30.818678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02cbe4-c07e-4ebb-90b7-2cb0f05de718 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:30.818972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02cbe4-c07e-4ebb-90b7-2cb0f05de718 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:30.819250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02cbe4-c07e-4ebb-90b7-2cb0f05de718 None admin] PATH_INFO: `/v3/domains/default/users/86f9463b664941f0b5c63b336b679b58/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:30.821450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02cbe4-c07e-4ebb-90b7-2cb0f05de718 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=86f9463b664941f0b5c63b336b679b58, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:30.823069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02cbe4-c07e-4ebb-90b7-2cb0f05de718 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:30.840144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 27/46] 10.222.0.71 () {58 vars in 1361 bytes} [Mon Jul 18 14:04:30 2022] PUT /identity/v3/domains/default/users/86f9463b664941f0b5c63b336b679b58/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:04:32.811241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51741a-f8cc-438f-a24c-58fce0f3737d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:32.811991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51741a-f8cc-438f-a24c-58fce0f3737d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:32.811991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51741a-f8cc-438f-a24c-58fce0f3737d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:32.813200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 20/47] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:32.818911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e9a41-4ac4-4460-9845-ec7b9dfc39c9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:32.819406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e9a41-4ac4-4460-9845-ec7b9dfc39c9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:32.819534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e9a41-4ac4-4460-9845-ec7b9dfc39c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:32.837147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-624e9a41-4ac4-4460-9845-ec7b9dfc39c9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:32.838385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-624e9a41-4ac4-4460-9845-ec7b9dfc39c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:32.911955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 28/48] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:32.936679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51741a-f8cc-438f-a24c-58fce0f3737d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:32.958358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df771c-f9ff-4acf-b798-0ed2ed226186 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:32.958597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df771c-f9ff-4acf-b798-0ed2ed226186 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:32.958901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df771c-f9ff-4acf-b798-0ed2ed226186 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:32.960304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 29/49] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:04:32.965521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910d778-1d79-4b04-9d3f-68cc94d2a051 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:32.965848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910d778-1d79-4b04-9d3f-68cc94d2a051 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:32.966171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910d778-1d79-4b04-9d3f-68cc94d2a051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:32.984380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5910d778-1d79-4b04-9d3f-68cc94d2a051 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:32.985616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5910d778-1d79-4b04-9d3f-68cc94d2a051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:33.015486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:33.061682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51741a-f8cc-438f-a24c-58fce0f3737d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:33.070991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 30/50] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:33.090442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:33.094612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910d778-1d79-4b04-9d3f-68cc94d2a051 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:33.109428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10fc46-bf5f-42e1-ba54-9393e4b2c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:33.110263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10fc46-bf5f-42e1-ba54-9393e4b2c775 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:33.110565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10fc46-bf5f-42e1-ba54-9393e4b2c775 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:33.110843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10fc46-bf5f-42e1-ba54-9393e4b2c775 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:33.112119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10fc46-bf5f-42e1-ba54-9393e4b2c775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:33.113618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10fc46-bf5f-42e1-ba54-9393e4b2c775 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:33.115325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 21/51] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:33.121492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10fc46-bf5f-42e1-ba54-9393e4b2c775 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:33.212388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910d778-1d79-4b04-9d3f-68cc94d2a051 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:33.227227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10fc46-bf5f-42e1-ba54-9393e4b2c775 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:33.248485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a4d18-3cdf-4366-89bf-abc3411390f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:33.249287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a4d18-3cdf-4366-89bf-abc3411390f7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:33.249564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a4d18-3cdf-4366-89bf-abc3411390f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:33.249868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a4d18-3cdf-4366-89bf-abc3411390f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:33.261166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a4d18-3cdf-4366-89bf-abc3411390f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:33.264320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4326c-1f7e-4bf9-a3da-0baaaad17bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:33.265005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4326c-1f7e-4bf9-a3da-0baaaad17bc9 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:33.265295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4326c-1f7e-4bf9-a3da-0baaaad17bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:33.265574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4326c-1f7e-4bf9-a3da-0baaaad17bc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:33.266340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4326c-1f7e-4bf9-a3da-0baaaad17bc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:33.267822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4326c-1f7e-4bf9-a3da-0baaaad17bc9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-916a4d18-3cdf-4366-89bf-abc3411390f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:33.269187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:33.270080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:33.270832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 31/52] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:04:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:04:33.276766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a4d18-3cdf-4366-89bf-abc3411390f7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:33.297494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 22/53] 10.222.0.71 () {60 vars in 1140 bytes} [Mon Jul 18 14:04:33 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:04:33.304085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:33.396673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a4d18-3cdf-4366-89bf-abc3411390f7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:33.438419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:33.439107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:33.439389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:33.439662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:33.440430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:33.440736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:33.442199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:33.451424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 32/54] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:33.457812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4326c-1f7e-4bf9-a3da-0baaaad17bc9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:33.502354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:33.561978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4326c-1f7e-4bf9-a3da-0baaaad17bc9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:33.595985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c06c1d-e79f-4387-b921-9c4cf4b60c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:33.596878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c06c1d-e79f-4387-b921-9c4cf4b60c49 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:33.597167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c06c1d-e79f-4387-b921-9c4cf4b60c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:33.597448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c06c1d-e79f-4387-b921-9c4cf4b60c49 None admin] PATH_INFO: `/v3/users/86f9463b664941f0b5c63b336b679b58` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:33.599005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c06c1d-e79f-4387-b921-9c4cf4b60c49 None admin] RBAC: Authorizing `identity:get_user(user_id=86f9463b664941f0b5c63b336b679b58)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:33.600499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c06c1d-e79f-4387-b921-9c4cf4b60c49 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:33.602217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 23/55] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:33 2022] GET /identity/v3/users/86f9463b664941f0b5c63b336b679b58 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:33.608079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:33.699402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:33.716681 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b1715-d88e-4c72-b30e-edd32ff162a0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:33.752961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae62117-270a-4171-95c3-73e3e5e658cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:33.753684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae62117-270a-4171-95c3-73e3e5e658cb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:33.753971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae62117-270a-4171-95c3-73e3e5e658cb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:33.754249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae62117-270a-4171-95c3-73e3e5e658cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:33.755477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae62117-270a-4171-95c3-73e3e5e658cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:33.756975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae62117-270a-4171-95c3-73e3e5e658cb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:33.758651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 33/56] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:33.765175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c06c1d-e79f-4387-b921-9c4cf4b60c49 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:33.983044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c06c1d-e79f-4387-b921-9c4cf4b60c49 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:33.992586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:34.019961 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9045091-397c-4f4a-871f-3461eecb99dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:34.020554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045091-397c-4f4a-871f-3461eecb99dd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:34.020842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045091-397c-4f4a-871f-3461eecb99dd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:34.021125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9045091-397c-4f4a-871f-3461eecb99dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:34.022005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9045091-397c-4f4a-871f-3461eecb99dd None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.domain.id=default, user.id=86f9463b664941f0b5c63b336b679b58) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:34.022325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9045091-397c-4f4a-871f-3461eecb99dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:34.023795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9045091-397c-4f4a-871f-3461eecb99dd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:34.032940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 24/57] 10.222.0.71 () {56 vars in 1322 bytes} [Mon Jul 18 14:04:33 2022] GET /identity/v3/role_assignments?user.id=86f9463b664941f0b5c63b336b679b58&scope.domain.id=default&role.id=5c672a1b2979444383c587a215adeb35 => generated 530 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:35.414705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc684ee-ba13-4b01-93d7-d43e73381423 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:35.415628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc684ee-ba13-4b01-93d7-d43e73381423 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:35.415628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc684ee-ba13-4b01-93d7-d43e73381423 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:35.418017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 34/58] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:35.427166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d8972-5108-4d7c-b0b2-5306318c8a96 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:35.427883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d8972-5108-4d7c-b0b2-5306318c8a96 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:35.428059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d8972-5108-4d7c-b0b2-5306318c8a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:35.454387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0c7d8972-5108-4d7c-b0b2-5306318c8a96 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:35.455922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0c7d8972-5108-4d7c-b0b2-5306318c8a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:35.534430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 25/59] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:35.558829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc684ee-ba13-4b01-93d7-d43e73381423 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:35.669269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc684ee-ba13-4b01-93d7-d43e73381423 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:35.705312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db34af-5724-491f-b3b6-031cf803cade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:35.706139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db34af-5724-491f-b3b6-031cf803cade None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:35.706423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db34af-5724-491f-b3b6-031cf803cade None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:35.706694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db34af-5724-491f-b3b6-031cf803cade None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:35.707928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db34af-5724-491f-b3b6-031cf803cade None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:35.709393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db34af-5724-491f-b3b6-031cf803cade None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:35.711041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 35/60] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:35.717050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d8972-5108-4d7c-b0b2-5306318c8a96 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:35.825277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d8972-5108-4d7c-b0b2-5306318c8a96 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:35.826837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:35.860453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc96b24-8a99-4ee7-b6fe-91fe9cdb4c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:35.861124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc96b24-8a99-4ee7-b6fe-91fe9cdb4c24 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:35.861407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc96b24-8a99-4ee7-b6fe-91fe9cdb4c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:35.861696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc96b24-8a99-4ee7-b6fe-91fe9cdb4c24 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:35.862428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc96b24-8a99-4ee7-b6fe-91fe9cdb4c24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:35.863915 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc96b24-8a99-4ee7-b6fe-91fe9cdb4c24 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:35.901338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 26/61] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:04:35 2022] POST /identity/v3/users => generated 275 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:39.289373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a351d2-a86f-4436-9c7c-4c84e9e3e6d7 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.290444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a351d2-a86f-4436-9c7c-4c84e9e3e6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.290930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a351d2-a86f-4436-9c7c-4c84e9e3e6d7 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.293171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 36/62] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:39.299127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fed77a-234f-4db1-99b3-3b92efbb47cd None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.300056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fed77a-234f-4db1-99b3-3b92efbb47cd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.300532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fed77a-234f-4db1-99b3-3b92efbb47cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.321826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f2fed77a-234f-4db1-99b3-3b92efbb47cd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:39.324152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f2fed77a-234f-4db1-99b3-3b92efbb47cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:39.457316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 27/63] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:39.501581 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a351d2-a86f-4436-9c7c-4c84e9e3e6d7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:39.536917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:39.538964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a75d5a-ae71-4169-ad93-31623b247fb8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.539582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a75d5a-ae71-4169-ad93-31623b247fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.540060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a75d5a-ae71-4169-ad93-31623b247fb8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.541869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 28/64] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:39.551916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cc42a-5da5-402a-a977-57b4144dd7bf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.552454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cc42a-5da5-402a-a977-57b4144dd7bf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.552985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46cc42a-5da5-402a-a977-57b4144dd7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.590849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a46cc42a-5da5-402a-a977-57b4144dd7bf None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:39.592097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a46cc42a-5da5-402a-a977-57b4144dd7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:39.652820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a351d2-a86f-4436-9c7c-4c84e9e3e6d7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:39.686704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 29/65] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:39.688477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e58bc-558a-4d89-8443-4816e38110e3 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.688994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e58bc-558a-4d89-8443-4816e38110e3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.689514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8e58bc-558a-4d89-8443-4816e38110e3 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.691105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 30/66] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:39.697312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7e6a3-c957-4aa5-9c91-beee22754434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:39.698470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7e6a3-c957-4aa5-9c91-beee22754434 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.699001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7e6a3-c957-4aa5-9c91-beee22754434 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.699498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7e6a3-c957-4aa5-9c91-beee22754434 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.703205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2094d5-58a6-4a06-9d46-c782ca1f58df None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.703766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2094d5-58a6-4a06-9d46-c782ca1f58df None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.704262 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2094d5-58a6-4a06-9d46-c782ca1f58df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.724686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6d2094d5-58a6-4a06-9d46-c782ca1f58df None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:39.726291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6d2094d5-58a6-4a06-9d46-c782ca1f58df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:39.726712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7e6a3-c957-4aa5-9c91-beee22754434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-43b7e6a3-c957-4aa5-9c91-beee22754434 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:39.734878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:39.735781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:39.736534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 37/67] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:39.737156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7e6a3-c957-4aa5-9c91-beee22754434 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:39.745604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:39.807481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 31/68] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:39.808748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2094d5-58a6-4a06-9d46-c782ca1f58df None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:39.848973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:39.902376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7e6a3-c957-4aa5-9c91-beee22754434 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:39.929217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2094d5-58a6-4a06-9d46-c782ca1f58df None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:39.941708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21221975-bb0b-4203-9fa9-75d65848e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:39.942861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221975-bb0b-4203-9fa9-75d65848e90c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.943328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221975-bb0b-4203-9fa9-75d65848e90c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.943824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21221975-bb0b-4203-9fa9-75d65848e90c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.958511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21221975-bb0b-4203-9fa9-75d65848e90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:39.966629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239f4ed-0433-41df-8bb9-a58c82c7665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:39.967311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f4ed-0433-41df-8bb9-a58c82c7665a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-21221975-bb0b-4203-9fa9-75d65848e90c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:39.967490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:39.968313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:39.969087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:39.969699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f4ed-0433-41df-8bb9-a58c82c7665a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.969699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239f4ed-0433-41df-8bb9-a58c82c7665a None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.969699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239f4ed-0433-41df-8bb9-a58c82c7665a None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:39.969699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239f4ed-0433-41df-8bb9-a58c82c7665a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:39.970439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239f4ed-0433-41df-8bb9-a58c82c7665a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:39.970714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 38/69] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:04:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:39.971788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21221975-bb0b-4203-9fa9-75d65848e90c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:39.979020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 32/70] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:39.980465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f12e-ab32-4a44-8d2a-0ece60be4608 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:39.980754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f12e-ab32-4a44-8d2a-0ece60be4608 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:39.981031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f12e-ab32-4a44-8d2a-0ece60be4608 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:39.982285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 33/71] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:39.983115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f12e-ab32-4a44-8d2a-0ece60be4608 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:39.998345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.083520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21221975-bb0b-4203-9fa9-75d65848e90c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.094923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f12e-ab32-4a44-8d2a-0ece60be4608 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.118626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e7d75-3979-4ff1-b7aa-2b0997564995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.119435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e7d75-3979-4ff1-b7aa-2b0997564995 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.119726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e7d75-3979-4ff1-b7aa-2b0997564995 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.120128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e7d75-3979-4ff1-b7aa-2b0997564995 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.128072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.130192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fec3d0-4f93-44ba-a645-5e148cf877c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.130855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fec3d0-4f93-44ba-a645-5e148cf877c4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.131143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fec3d0-4f93-44ba-a645-5e148cf877c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.131419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fec3d0-4f93-44ba-a645-5e148cf877c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.131589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e7d75-3979-4ff1-b7aa-2b0997564995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:40.132179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fec3d0-4f93-44ba-a645-5e148cf877c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:40.132492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fec3d0-4f93-44ba-a645-5e148cf877c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.133922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fec3d0-4f93-44ba-a645-5e148cf877c4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-281e7d75-3979-4ff1-b7aa-2b0997564995 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:40.139591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:40.140505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 39/72] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:40.144423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e7d75-3979-4ff1-b7aa-2b0997564995 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.145299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 34/73] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:04:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:40.145299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5332d0-8e98-4db0-ad2a-e6eea27c85ad None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.145299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5332d0-8e98-4db0-ad2a-e6eea27c85ad None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.145299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5332d0-8e98-4db0-ad2a-e6eea27c85ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.162661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fb5332d0-8e98-4db0-ad2a-e6eea27c85ad None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:40.163904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fb5332d0-8e98-4db0-ad2a-e6eea27c85ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:40.232404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 35/74] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:40.233557 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07803a3a-c67f-4ffb-9c39-d4698f9b8b63 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.233885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07803a3a-c67f-4ffb-9c39-d4698f9b8b63 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.234167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07803a3a-c67f-4ffb-9c39-d4698f9b8b63 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.235614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 36/75] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:40.236171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07803a3a-c67f-4ffb-9c39-d4698f9b8b63 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.239146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.259565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e7d75-3979-4ff1-b7aa-2b0997564995 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.297199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41059685-daf4-45de-ba8a-916374310266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.298127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41059685-daf4-45de-ba8a-916374310266 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.298418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41059685-daf4-45de-ba8a-916374310266 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.298703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41059685-daf4-45de-ba8a-916374310266 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.314052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41059685-daf4-45de-ba8a-916374310266 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.315564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41059685-daf4-45de-ba8a-916374310266 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.317309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 40/76] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:40.318135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41059685-daf4-45de-ba8a-916374310266 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.353120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07803a3a-c67f-4ffb-9c39-d4698f9b8b63 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.382933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.391663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa7ab2-cbed-4fdc-ab44-92c52f6789a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.392591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa7ab2-cbed-4fdc-ab44-92c52f6789a5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.392877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa7ab2-cbed-4fdc-ab44-92c52f6789a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.393154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa7ab2-cbed-4fdc-ab44-92c52f6789a5 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.394464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa7ab2-cbed-4fdc-ab44-92c52f6789a5 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.395969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa7ab2-cbed-4fdc-ab44-92c52f6789a5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.397664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 37/77] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:40.398918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d7c13-48ca-41d6-83f4-29af08f359cb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.399211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d7c13-48ca-41d6-83f4-29af08f359cb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.399490 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d7c13-48ca-41d6-83f4-29af08f359cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.422904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b68d7c13-48ca-41d6-83f4-29af08f359cb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:40.424587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b68d7c13-48ca-41d6-83f4-29af08f359cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:40.441285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41059685-daf4-45de-ba8a-916374310266 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.478597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.479340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.479626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.479911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.480735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:40.480964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.482410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.490939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 41/78] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:40.491740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.494234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.500871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 38/79] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:40.502018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d7c13-48ca-41d6-83f4-29af08f359cb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.593759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52725642-2a0f-48eb-81ff-e98be7b1dad8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.604423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d7c13-48ca-41d6-83f4-29af08f359cb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.613556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.629045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3197e-9e02-4413-87f0-feaead475749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.629888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3197e-9e02-4413-87f0-feaead475749 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.630172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3197e-9e02-4413-87f0-feaead475749 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.630449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3197e-9e02-4413-87f0-feaead475749 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.642233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3197e-9e02-4413-87f0-feaead475749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:40.647898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c0e6e-732b-4183-98d1-56c2640b1e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.648565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c0e6e-732b-4183-98d1-56c2640b1e60 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.648856 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c0e6e-732b-4183-98d1-56c2640b1e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.649139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c0e6e-732b-4183-98d1-56c2640b1e60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-19d3197e-9e02-4413-87f0-feaead475749 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:40.649929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:40.650750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 42/80] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:40.652649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3197e-9e02-4413-87f0-feaead475749 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.717974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.720927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c0e6e-732b-4183-98d1-56c2640b1e60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.722955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c0e6e-732b-4183-98d1-56c2640b1e60 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.756421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3197e-9e02-4413-87f0-feaead475749 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.791088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c4f60-85d7-420a-b1e3-0b5695a494e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.791791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c4f60-85d7-420a-b1e3-0b5695a494e8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.792078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c4f60-85d7-420a-b1e3-0b5695a494e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.792464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c4f60-85d7-420a-b1e3-0b5695a494e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.823616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 39/81] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:40.824266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c0e6e-732b-4183-98d1-56c2640b1e60 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:40.847280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.878669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c4f60-85d7-420a-b1e3-0b5695a494e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.880679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c4f60-85d7-420a-b1e3-0b5695a494e8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.934971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:40.938545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c0e6e-732b-4183-98d1-56c2640b1e60 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:40.980716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ef7e-5384-4ea3-b8f4-d4deed5c37fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:40.981758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ef7e-5384-4ea3-b8f4-d4deed5c37fe None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:40.982118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ef7e-5384-4ea3-b8f4-d4deed5c37fe None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:40.982472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5ef7e-5384-4ea3-b8f4-d4deed5c37fe None admin] PATH_INFO: `/v3/users/86f9463b664941f0b5c63b336b679b58` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:40.983787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ef7e-5384-4ea3-b8f4-d4deed5c37fe None admin] RBAC: Authorizing `identity:get_user(user_id=86f9463b664941f0b5c63b336b679b58)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:40.985329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5ef7e-5384-4ea3-b8f4-d4deed5c37fe None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:40.987098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 40/82] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/users/86f9463b664941f0b5c63b336b679b58 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:40.992566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ef7e-5384-4ea3-b8f4-d4deed5c37fe None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.004166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 43/83] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:41.005328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c4f60-85d7-420a-b1e3-0b5695a494e8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.090812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:41.102753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5ef7e-5384-4ea3-b8f4-d4deed5c37fe None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.111626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c4f60-85d7-420a-b1e3-0b5695a494e8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.138072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710da3a3-efb6-4f1e-a74c-4ff33502d2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.138803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da3a3-efb6-4f1e-a74c-4ff33502d2ec None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.139093 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da3a3-efb6-4f1e-a74c-4ff33502d2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.139370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710da3a3-efb6-4f1e-a74c-4ff33502d2ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.140587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710da3a3-efb6-4f1e-a74c-4ff33502d2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.142090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710da3a3-efb6-4f1e-a74c-4ff33502d2ec None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.143901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 41/84] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:41.144516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710da3a3-efb6-4f1e-a74c-4ff33502d2ec None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.155866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.157412 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.157959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.158772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.160588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:41.161138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.164424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.174146 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 44/85] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:41.175053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.183672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:41.251862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710da3a3-efb6-4f1e-a74c-4ff33502d2ec None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.281288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c589-bd38-4663-9c9d-7a5e9d54a9f7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.287438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73364045-1920-481c-a713-74bf272f3a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.288314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364045-1920-481c-a713-74bf272f3a09 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.288608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364045-1920-481c-a713-74bf272f3a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.288888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73364045-1920-481c-a713-74bf272f3a09 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.290139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364045-1920-481c-a713-74bf272f3a09 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.291655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73364045-1920-481c-a713-74bf272f3a09 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.293400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 42/86] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:41.294218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73364045-1920-481c-a713-74bf272f3a09 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.319595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b1b7-3c8a-4d3f-85a4-8b1492612e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.320452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b1b7-3c8a-4d3f-85a4-8b1492612e1d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.320833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b1b7-3c8a-4d3f-85a4-8b1492612e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.321207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1b1b7-3c8a-4d3f-85a4-8b1492612e1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.333995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:41.391747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1b1b7-3c8a-4d3f-85a4-8b1492612e1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.393145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1b1b7-3c8a-4d3f-85a4-8b1492612e1d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.399494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73364045-1920-481c-a713-74bf272f3a09 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.402668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:41.436125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e435-67f1-48d4-9979-3f210675af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.437007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e435-67f1-48d4-9979-3f210675af22 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.437288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e435-67f1-48d4-9979-3f210675af22 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.437566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76e435-67f1-48d4-9979-3f210675af22 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.439092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76e435-67f1-48d4-9979-3f210675af22 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.440992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76e435-67f1-48d4-9979-3f210675af22 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.442793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 43/87] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:41.443561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e435-67f1-48d4-9979-3f210675af22 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.501007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 45/88] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:41.501894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b1b7-3c8a-4d3f-85a4-8b1492612e1d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.551990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:41.561573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76e435-67f1-48d4-9979-3f210675af22 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.600775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1a60c-a259-4193-8fb3-b7043da68e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.601599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a60c-a259-4193-8fb3-b7043da68e06 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.601912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a60c-a259-4193-8fb3-b7043da68e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.602195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a60c-a259-4193-8fb3-b7043da68e06 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.602950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1a60c-a259-4193-8fb3-b7043da68e06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.604858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1a60c-a259-4193-8fb3-b7043da68e06 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.614218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1b1b7-3c8a-4d3f-85a4-8b1492612e1d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.632707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 44/89] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:04:41 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:41.632707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1a60c-a259-4193-8fb3-b7043da68e06 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.671565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189c890-0e62-430d-a6d9-2f7719898d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.672376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189c890-0e62-430d-a6d9-2f7719898d84 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.672652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189c890-0e62-430d-a6d9-2f7719898d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.672931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189c890-0e62-430d-a6d9-2f7719898d84 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.674249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189c890-0e62-430d-a6d9-2f7719898d84 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.675760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189c890-0e62-430d-a6d9-2f7719898d84 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.676554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:41.682169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 46/90] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:41.682169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189c890-0e62-430d-a6d9-2f7719898d84 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.769186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1a60c-a259-4193-8fb3-b7043da68e06 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.799533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189c890-0e62-430d-a6d9-2f7719898d84 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:41.811833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:41.815727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.816604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.816892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.817171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.818090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] RBAC: Adding query filter params (role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb, user.id=780b62fec93b4ec58e2b4cb92e229771) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:41.818415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.819890 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.826320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 45/91] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/role_assignments?user.id=780b62fec93b4ec58e2b4cb92e229771&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:41.829707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.855199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:41.856405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:41.856702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:41.856986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:41.859001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] RBAC: Adding query filter params (role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb, user.id=780b62fec93b4ec58e2b4cb92e229771) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:41.859338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:41.861035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:41.869691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 47/92] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/role_assignments?user.id=780b62fec93b4ec58e2b4cb92e229771&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:41.877825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:41.966201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:41.973573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91708d-5fb3-4052-8732-1e5168238a46 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:42.004149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4934ad-8385-4f4a-b68c-76c0e0194810 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:42.031612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b14c6-eb06-4fdf-bbcf-084a00fae887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:42.032547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b14c6-eb06-4fdf-bbcf-084a00fae887 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:42.032843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b14c6-eb06-4fdf-bbcf-084a00fae887 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:42.033215 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b14c6-eb06-4fdf-bbcf-084a00fae887 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:42.036119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b14c6-eb06-4fdf-bbcf-084a00fae887 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:42.037802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b14c6-eb06-4fdf-bbcf-084a00fae887 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:42.039708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 46/93] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:42.046454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b14c6-eb06-4fdf-bbcf-084a00fae887 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:42.050839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf03046-e567-4da6-90f3-9824c94a3f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:42.051526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03046-e567-4da6-90f3-9824c94a3f51 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:42.051814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03046-e567-4da6-90f3-9824c94a3f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:42.052095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf03046-e567-4da6-90f3-9824c94a3f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:42.107645 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:42.139603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf03046-e567-4da6-90f3-9824c94a3f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:42.141281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf03046-e567-4da6-90f3-9824c94a3f51 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:42.186203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b14c6-eb06-4fdf-bbcf-084a00fae887 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:42.259370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:42.259970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:42.260254 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:42.260870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:42.263452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb, user.id=86f9463b664941f0b5c63b336b679b58) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:42.263780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:42.265848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:42.274890 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 47/94] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:42 2022] GET /identity/v3/role_assignments?user.id=86f9463b664941f0b5c63b336b679b58&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=5c672a1b2979444383c587a215adeb35 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:42.303665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 48/95] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:42.310211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:42.350327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:42.460004 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bd5bd-510b-4d29-9cef-dc807f0debcb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:42.513527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30332cf8-dd67-44f0-811f-c20afc3e71c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:42.514379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30332cf8-dd67-44f0-811f-c20afc3e71c6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:42.514672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30332cf8-dd67-44f0-811f-c20afc3e71c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:42.514953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30332cf8-dd67-44f0-811f-c20afc3e71c6 None admin] PATH_INFO: `/v3/projects/17a73495723046079447956b439eb150` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:42.516285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30332cf8-dd67-44f0-811f-c20afc3e71c6 None admin] RBAC: Authorizing `identity:get_project(project_id=17a73495723046079447956b439eb150)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:42.517915 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30332cf8-dd67-44f0-811f-c20afc3e71c6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:42.519758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 48/96] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:42 2022] GET /identity/v3/projects/17a73495723046079447956b439eb150 => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:42.525851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf03046-e567-4da6-90f3-9824c94a3f51 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:42.633485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:42.673362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf03046-e567-4da6-90f3-9824c94a3f51 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:42.717330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4fc42-7b2a-4a54-b595-91c9fb343312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:42.718401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4fc42-7b2a-4a54-b595-91c9fb343312 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:42.718883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4fc42-7b2a-4a54-b595-91c9fb343312 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:42.719380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4fc42-7b2a-4a54-b595-91c9fb343312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:42.720419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4fc42-7b2a-4a54-b595-91c9fb343312 None admin] RBAC: Adding query filter params (role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=17a73495723046079447956b439eb150, user.id=780b62fec93b4ec58e2b4cb92e229771) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:42.720934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4fc42-7b2a-4a54-b595-91c9fb343312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:42.722757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4fc42-7b2a-4a54-b595-91c9fb343312 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:42.730052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 49/97] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:42 2022] GET /identity/v3/role_assignments?user.id=780b62fec93b4ec58e2b4cb92e229771&scope.project.id=17a73495723046079447956b439eb150&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:44.609290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721450e-f599-4d3c-817f-2cf582308ee0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:44.610383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721450e-f599-4d3c-817f-2cf582308ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:44.610869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721450e-f599-4d3c-817f-2cf582308ee0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:44.612488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 49/98] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:44.623995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e91b-5705-4931-b23c-e91a67c69b0a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:44.624957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e91b-5705-4931-b23c-e91a67c69b0a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:44.625528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319e91b-5705-4931-b23c-e91a67c69b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:44.658778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6319e91b-5705-4931-b23c-e91a67c69b0a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:44.660019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6319e91b-5705-4931-b23c-e91a67c69b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:44.754608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 50/99] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:44.789811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721450e-f599-4d3c-817f-2cf582308ee0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:44.833906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585ed31-eec0-4e19-a200-7cfd2fb8f396 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:44.834224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585ed31-eec0-4e19-a200-7cfd2fb8f396 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:44.834471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585ed31-eec0-4e19-a200-7cfd2fb8f396 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:44.835871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 51/100] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:04:44.841400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3213792-8ef6-4e17-bc7c-dfe149871ff1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:44.841705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3213792-8ef6-4e17-bc7c-dfe149871ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:44.841997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3213792-8ef6-4e17-bc7c-dfe149871ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:44.865265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a3213792-8ef6-4e17-bc7c-dfe149871ff1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:44.866415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a3213792-8ef6-4e17-bc7c-dfe149871ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:44.946036 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:44.946669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:44.956538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 52/101] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:44.959143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66291054-cd52-40df-8838-b9f8e61c4d8a None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:44.959773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66291054-cd52-40df-8838-b9f8e61c4d8a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:44.961018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66291054-cd52-40df-8838-b9f8e61c4d8a None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:44.963949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 53/102] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:44.966003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2721450e-f599-4d3c-817f-2cf582308ee0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:44.969747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989b3f-898b-4a0d-8048-9386d55b6952 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:44.970224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989b3f-898b-4a0d-8048-9386d55b6952 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:44.970698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28989b3f-898b-4a0d-8048-9386d55b6952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.000807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-28989b3f-898b-4a0d-8048-9386d55b6952 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:45.003531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-28989b3f-898b-4a0d-8048-9386d55b6952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:45.038902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794523d8-3a01-43f0-baf8-0ce07346eb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:45.040028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794523d8-3a01-43f0-baf8-0ce07346eb28 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.040516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794523d8-3a01-43f0-baf8-0ce07346eb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.040964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794523d8-3a01-43f0-baf8-0ce07346eb28 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.064226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794523d8-3a01-43f0-baf8-0ce07346eb28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-794523d8-3a01-43f0-baf8-0ce07346eb28 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:45.084541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:45.085510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:45.086280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 50/103] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:45.086919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794523d8-3a01-43f0-baf8-0ce07346eb28 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:45.103878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 54/104] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:45.105679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989b3f-898b-4a0d-8048-9386d55b6952 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:45.221704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28989b3f-898b-4a0d-8048-9386d55b6952 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:45.261075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:45.262359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.262962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.263436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.264474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:45.264981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:45.266098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794523d8-3a01-43f0-baf8-0ce07346eb28 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:45.266891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:45.277688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 55/105] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:45.278278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:45.287302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:45.308238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63fe23-7ff5-466e-98b5-b3748490e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:45.309286 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63fe23-7ff5-466e-98b5-b3748490e20e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.309579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63fe23-7ff5-466e-98b5-b3748490e20e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.310698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63fe23-7ff5-466e-98b5-b3748490e20e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.312767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63fe23-7ff5-466e-98b5-b3748490e20e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:45.315336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63fe23-7ff5-466e-98b5-b3748490e20e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:45.317247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 51/106] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:45.319845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63fe23-7ff5-466e-98b5-b3748490e20e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:45.348154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:45.434747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5399399c-0116-4696-8179-52b4fb5946f7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:45.444653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63fe23-7ff5-466e-98b5-b3748490e20e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:45.477330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ffe99-31a9-44d5-8fec-85271ee96e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:45.478394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffe99-31a9-44d5-8fec-85271ee96e09 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.478782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffe99-31a9-44d5-8fec-85271ee96e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.479258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffe99-31a9-44d5-8fec-85271ee96e09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.484745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c889d-331c-4d87-9aad-df0978983f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:45.485995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c889d-331c-4d87-9aad-df0978983f26 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.486553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c889d-331c-4d87-9aad-df0978983f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.487046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c889d-331c-4d87-9aad-df0978983f26 None admin] PATH_INFO: `/v3/users/0d26578e1ae649f1ac2c3902b3cb25ad` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.492297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ffe99-31a9-44d5-8fec-85271ee96e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-4e2ffe99-31a9-44d5-8fec-85271ee96e09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:45.502315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:45.503218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:45.504074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 56/107] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:04:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:45.504701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ffe99-31a9-44d5-8fec-85271ee96e09 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:45.508152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c889d-331c-4d87-9aad-df0978983f26 None admin] RBAC: Authorizing `identity:get_user(user_id=0d26578e1ae649f1ac2c3902b3cb25ad)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:45.510392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c889d-331c-4d87-9aad-df0978983f26 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:45.512758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 52/108] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:45 2022] GET /identity/v3/users/0d26578e1ae649f1ac2c3902b3cb25ad => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:45.514277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6771e3b-233e-4379-8cbf-5cca66d06de0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.514792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6771e3b-233e-4379-8cbf-5cca66d06de0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.515327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6771e3b-233e-4379-8cbf-5cca66d06de0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.517453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 53/109] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:45.518624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6771e3b-233e-4379-8cbf-5cca66d06de0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:45.565805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:45.613631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:45.626610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ffe99-31a9-44d5-8fec-85271ee96e09 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:45.645827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6771e3b-233e-4379-8cbf-5cca66d06de0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:45.677027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffafb3-ddef-4b71-a5c9-d3dc39bec400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:45.678586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffafb3-ddef-4b71-a5c9-d3dc39bec400 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.679152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffafb3-ddef-4b71-a5c9-d3dc39bec400 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.679579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffafb3-ddef-4b71-a5c9-d3dc39bec400 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.680945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffafb3-ddef-4b71-a5c9-d3dc39bec400 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:45.683448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ffafb3-ddef-4b71-a5c9-d3dc39bec400 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:45.688362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:45.689038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.689322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.689593 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.690386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:45.690698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:45.692067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:45.700815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 54/110] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:04:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:45.701560 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:45.722152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 57/111] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:04:45 2022] POST /identity/v3/projects => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:45.723498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebcf23-d757-4106-b7d7-3be7fc79f6b8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.724053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebcf23-d757-4106-b7d7-3be7fc79f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.724053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebcf23-d757-4106-b7d7-3be7fc79f6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.742428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9aebcf23-d757-4106-b7d7-3be7fc79f6b8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:45.743898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9aebcf23-d757-4106-b7d7-3be7fc79f6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:45.839076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 58/112] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:45.840696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329717-3b0f-4083-9345-e73a913e8d0c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.840987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329717-3b0f-4083-9345-e73a913e8d0c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.841261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09329717-3b0f-4083-9345-e73a913e8d0c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.842673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 59/113] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:45.847803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09329717-3b0f-4083-9345-e73a913e8d0c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:45.854452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcdcbda-6b0b-420e-9241-4073b1f66841 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:45.889534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:45.898332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d8bbf-1c23-407d-8eeb-1275a0adb0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:45.899001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d8bbf-1c23-407d-8eeb-1275a0adb0ff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:45.899289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d8bbf-1c23-407d-8eeb-1275a0adb0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:45.899617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d8bbf-1c23-407d-8eeb-1275a0adb0ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:45.912360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:45.990975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d8bbf-1c23-407d-8eeb-1275a0adb0ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:45.992782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d8bbf-1c23-407d-8eeb-1275a0adb0ff None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:45.992896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09329717-3b0f-4083-9345-e73a913e8d0c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:46.030112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129e4e4-ee80-45cc-85fa-f02c00fb9eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:46.031135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e4e4-ee80-45cc-85fa-f02c00fb9eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.031425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e4e4-ee80-45cc-85fa-f02c00fb9eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.031704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129e4e4-ee80-45cc-85fa-f02c00fb9eb7 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.033199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129e4e4-ee80-45cc-85fa-f02c00fb9eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:46.034725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7129e4e4-ee80-45cc-85fa-f02c00fb9eb7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:46.037595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 60/114] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:45 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:46.039009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474e0be-f338-4fc1-abf0-22a1786f65c3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.039168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474e0be-f338-4fc1-abf0-22a1786f65c3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.039431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474e0be-f338-4fc1-abf0-22a1786f65c3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.043632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 61/115] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:46.043632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c82a7-491f-4aab-a2a4-f01594ae4218 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.043632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c82a7-491f-4aab-a2a4-f01594ae4218 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.043632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c82a7-491f-4aab-a2a4-f01594ae4218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.064117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-653c82a7-491f-4aab-a2a4-f01594ae4218 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:46.065430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-653c82a7-491f-4aab-a2a4-f01594ae4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:46.118617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 55/116] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:46.118617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d8bbf-1c23-407d-8eeb-1275a0adb0ff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.141452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 62/117] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:46.147261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c82a7-491f-4aab-a2a4-f01594ae4218 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.164566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:46.192912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:46.234251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d8bbf-1c23-407d-8eeb-1275a0adb0ff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:46.273412 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c82a7-491f-4aab-a2a4-f01594ae4218 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:46.292476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da72da-2ac3-46c6-a065-f0b8fe7ada00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:46.293358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da72da-2ac3-46c6-a065-f0b8fe7ada00 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.293647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da72da-2ac3-46c6-a065-f0b8fe7ada00 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.293952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da72da-2ac3-46c6-a065-f0b8fe7ada00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.304989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da72da-2ac3-46c6-a065-f0b8fe7ada00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a1da72da-2ac3-46c6-a065-f0b8fe7ada00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:46.312881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:46.313874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:46.314641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 56/118] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:46.315543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b7fbe-b400-463f-b795-760839188831 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.315787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b7fbe-b400-463f-b795-760839188831 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.316082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b7fbe-b400-463f-b795-760839188831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.329273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00aae36-0dea-477d-aef6-03dc1ee75536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:46.330243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00aae36-0dea-477d-aef6-03dc1ee75536 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.330532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00aae36-0dea-477d-aef6-03dc1ee75536 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.330814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00aae36-0dea-477d-aef6-03dc1ee75536 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.332062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00aae36-0dea-477d-aef6-03dc1ee75536 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:46.333576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00aae36-0dea-477d-aef6-03dc1ee75536 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:46.335389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 63/119] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:46 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:46.336191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00aae36-0dea-477d-aef6-03dc1ee75536 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.348742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6f4b7fbe-b400-463f-b795-760839188831 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:46.350658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6f4b7fbe-b400-463f-b795-760839188831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:46.450398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 57/120] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:46.451367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b7fbe-b400-463f-b795-760839188831 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.479301 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:46.483688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00aae36-0dea-477d-aef6-03dc1ee75536 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:46.497119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:46.526490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760338-869d-4643-8066-ab04829e1d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:46.527789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760338-869d-4643-8066-ab04829e1d23 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.528070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760338-869d-4643-8066-ab04829e1d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.528352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16760338-869d-4643-8066-ab04829e1d23 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.529591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16760338-869d-4643-8066-ab04829e1d23 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:46.531094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16760338-869d-4643-8066-ab04829e1d23 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:46.537502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 64/121] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:46 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:46.537502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760338-869d-4643-8066-ab04829e1d23 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.579563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b7fbe-b400-463f-b795-760839188831 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:46.619166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b76c1e-a747-4b75-b33a-cc86ae598303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:46.620423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b76c1e-a747-4b75-b33a-cc86ae598303 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.620766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b76c1e-a747-4b75-b33a-cc86ae598303 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.621057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b76c1e-a747-4b75-b33a-cc86ae598303 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.637566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b76c1e-a747-4b75-b33a-cc86ae598303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:46.643420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16760338-869d-4643-8066-ab04829e1d23 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-56b76c1e-a747-4b75-b33a-cc86ae598303 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:46.643898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:46.644680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:46.645418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 58/122] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:04:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:46.646981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b76c1e-a747-4b75-b33a-cc86ae598303 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.685480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:46.686485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.686953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.687218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.688006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:46.688319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:46.689738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:46.699181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 65/123] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:46.699790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.730658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:46.755872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:46.767296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b76c1e-a747-4b75-b33a-cc86ae598303 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:46.805758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dcaaf-0b61-46fe-a74a-b8922f5690a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:46.806724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dcaaf-0b61-46fe-a74a-b8922f5690a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.806970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dcaaf-0b61-46fe-a74a-b8922f5690a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.807295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dcaaf-0b61-46fe-a74a-b8922f5690a4 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/users/780b62fec93b4ec58e2b4cb92e229771/roles/89d5f9aaa6284371b4d5205fea92cba1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.809887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dcaaf-0b61-46fe-a74a-b8922f5690a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=43134d25fcbf4ee6a4e0b977b35eddeb, user_id=780b62fec93b4ec58e2b4cb92e229771, role_id=89d5f9aaa6284371b4d5205fea92cba1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:46.811171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112cf79c-fdb1-46bc-bb35-99d266c0def9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:46.811391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dcaaf-0b61-46fe-a74a-b8922f5690a4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:46.830403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 59/124] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:04:46 2022] PUT /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/users/780b62fec93b4ec58e2b4cb92e229771/roles/89d5f9aaa6284371b4d5205fea92cba1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:04:46.831031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dcaaf-0b61-46fe-a74a-b8922f5690a4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.863911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272f5a3-b73b-4a73-993e-c58021aee000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:46.865241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272f5a3-b73b-4a73-993e-c58021aee000 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:46.865782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272f5a3-b73b-4a73-993e-c58021aee000 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:46.866294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272f5a3-b73b-4a73-993e-c58021aee000 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:46.881456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272f5a3-b73b-4a73-993e-c58021aee000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c272f5a3-b73b-4a73-993e-c58021aee000 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:46.891626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:46.892589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:46.893392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:46.896667 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 66/125] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:46.897926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272f5a3-b73b-4a73-993e-c58021aee000 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:46.964506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dcaaf-0b61-46fe-a74a-b8922f5690a4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.004288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.004995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.005283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.005564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.006470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] RBAC: Adding query filter params (role.id=6e98f7101fb9430a87421182e7b6f4a8, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb, user.id=0d26578e1ae649f1ac2c3902b3cb25ad) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:47.006885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.008401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.014803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 60/126] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:46 2022] GET /identity/v3/role_assignments?user.id=0d26578e1ae649f1ac2c3902b3cb25ad&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=6e98f7101fb9430a87421182e7b6f4a8 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:47.015760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.016561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:47.032836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272f5a3-b73b-4a73-993e-c58021aee000 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.038989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:47.095265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.096535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.098118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.099211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.100549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:47.101240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.104964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.125316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 67/127] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:47.128058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.213696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606375fd-ef78-498f-95ca-16fe3953eb1a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.253936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29babea4-e54a-4844-8dc6-aec86ed637f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.255216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29babea4-e54a-4844-8dc6-aec86ed637f8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.255698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29babea4-e54a-4844-8dc6-aec86ed637f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.256201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29babea4-e54a-4844-8dc6-aec86ed637f8 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.257711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29babea4-e54a-4844-8dc6-aec86ed637f8 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.259649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29babea4-e54a-4844-8dc6-aec86ed637f8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.262439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 61/128] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:47 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:47.263473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29babea4-e54a-4844-8dc6-aec86ed637f8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.293525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f04a8f-c984-4e51-9d48-1e5373be9d46 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.359020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.360121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.360620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.361101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.362100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:47.362617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.364249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.375506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 68/129] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:47.376309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:47.377524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.395059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:47.418720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29babea4-e54a-4844-8dc6-aec86ed637f8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.511116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e7286-0c6a-45e2-b127-e62893a6f7a1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.514798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5cf02-4432-46f0-9917-7703a13ee8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.515349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5cf02-4432-46f0-9917-7703a13ee8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.515639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5cf02-4432-46f0-9917-7703a13ee8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.515931 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5cf02-4432-46f0-9917-7703a13ee8a0 None admin] PATH_INFO: `/v3/users/86f9463b664941f0b5c63b336b679b58` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.517238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5cf02-4432-46f0-9917-7703a13ee8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=86f9463b664941f0b5c63b336b679b58)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.518762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5cf02-4432-46f0-9917-7703a13ee8a0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.520465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 62/130] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:47 2022] GET /identity/v3/users/86f9463b664941f0b5c63b336b679b58 => generated 247 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:47.521264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5cf02-4432-46f0-9917-7703a13ee8a0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.557877 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf61b-11a0-4077-a650-a46a9a461dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.559208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf61b-11a0-4077-a650-a46a9a461dda None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.560682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf61b-11a0-4077-a650-a46a9a461dda None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.561187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf61b-11a0-4077-a650-a46a9a461dda None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.563775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaf61b-11a0-4077-a650-a46a9a461dda None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.566904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaf61b-11a0-4077-a650-a46a9a461dda None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.569265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 69/131] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:47 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:47.570272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf61b-11a0-4077-a650-a46a9a461dda None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.660072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:47.683678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf61b-11a0-4077-a650-a46a9a461dda None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.707644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5cf02-4432-46f0-9917-7703a13ee8a0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.720657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cae28-10b0-40b6-b1e7-f7722f219f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.721898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cae28-10b0-40b6-b1e7-f7722f219f72 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.722410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cae28-10b0-40b6-b1e7-f7722f219f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.722902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cae28-10b0-40b6-b1e7-f7722f219f72 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.724285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cae28-10b0-40b6-b1e7-f7722f219f72 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.725984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cae28-10b0-40b6-b1e7-f7722f219f72 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.728030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 70/132] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:47 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:47.728973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cae28-10b0-40b6-b1e7-f7722f219f72 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.795051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13409c7f-8fd7-4590-9672-5a0d3e13ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.795982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13409c7f-8fd7-4590-9672-5a0d3e13ca29 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.796274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13409c7f-8fd7-4590-9672-5a0d3e13ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.796556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13409c7f-8fd7-4590-9672-5a0d3e13ca29 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.802608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13409c7f-8fd7-4590-9672-5a0d3e13ca29 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.804140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13409c7f-8fd7-4590-9672-5a0d3e13ca29 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.805969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 63/133] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:47 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:47.806758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13409c7f-8fd7-4590-9672-5a0d3e13ca29 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.840455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cae28-10b0-40b6-b1e7-f7722f219f72 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:47.858172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:47.879606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0582b5-02ab-46e3-a07d-4810b0531b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:47.880525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0582b5-02ab-46e3-a07d-4810b0531b99 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:47.880823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0582b5-02ab-46e3-a07d-4810b0531b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:47.881108 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0582b5-02ab-46e3-a07d-4810b0531b99 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/users/780b62fec93b4ec58e2b4cb92e229771/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:47.884634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0582b5-02ab-46e3-a07d-4810b0531b99 None admin] RBAC: Authorizing `identity:create_grant(project_id=43134d25fcbf4ee6a4e0b977b35eddeb, user_id=780b62fec93b4ec58e2b4cb92e229771, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:47.886427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0582b5-02ab-46e3-a07d-4810b0531b99 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:47.913282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 71/134] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:04:47 2022] PUT /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/users/780b62fec93b4ec58e2b4cb92e229771/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:04:47.914057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0582b5-02ab-46e3-a07d-4810b0531b99 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:47.934831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:48.028408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13409c7f-8fd7-4590-9672-5a0d3e13ca29 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:48.090294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d60092-013c-4fa1-98c6-1b1c72444c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:48.093252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60092-013c-4fa1-98c6-1b1c72444c8d None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:48.095137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60092-013c-4fa1-98c6-1b1c72444c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:48.096332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60092-013c-4fa1-98c6-1b1c72444c8d None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/users/86f9463b664941f0b5c63b336b679b58/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:48.100852 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d60092-013c-4fa1-98c6-1b1c72444c8d None admin] RBAC: Authorizing `identity:create_grant(project_id=43134d25fcbf4ee6a4e0b977b35eddeb, user_id=86f9463b664941f0b5c63b336b679b58, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:48.103301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d60092-013c-4fa1-98c6-1b1c72444c8d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:48.112810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0582b5-02ab-46e3-a07d-4810b0531b99 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:48.128956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 64/135] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:04:47 2022] PUT /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/users/86f9463b664941f0b5c63b336b679b58/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:04:48.131706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c67760-876a-41f2-b3ed-587d90bb7740 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:48.133678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c67760-876a-41f2-b3ed-587d90bb7740 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:48.134195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c67760-876a-41f2-b3ed-587d90bb7740 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:48.138875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 65/136] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:48.143757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9fa7-add0-4fe6-a3bc-bc164d34c776 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:48.144330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9fa7-add0-4fe6-a3bc-bc164d34c776 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:48.144863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc9fa7-add0-4fe6-a3bc-bc164d34c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:48.171848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e9bc9fa7-add0-4fe6-a3bc-bc164d34c776 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:48.173098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e9bc9fa7-add0-4fe6-a3bc-bc164d34c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:48.177994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f86c-968b-43fc-813e-89194a79d87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:48.179198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f86c-968b-43fc-813e-89194a79d87e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:48.179751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f86c-968b-43fc-813e-89194a79d87e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:48.180403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f86c-968b-43fc-813e-89194a79d87e None admin] PATH_INFO: `/v3/projects/17a73495723046079447956b439eb150` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:48.182313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f86c-968b-43fc-813e-89194a79d87e None admin] RBAC: Authorizing `identity:get_project(project_id=17a73495723046079447956b439eb150)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:48.184154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f86c-968b-43fc-813e-89194a79d87e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:48.187176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 72/137] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:47 2022] GET /identity/v3/projects/17a73495723046079447956b439eb150 => generated 312 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:48.194713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f86c-968b-43fc-813e-89194a79d87e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:48.280284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 66/138] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:48.301848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:48.315201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc9fa7-add0-4fe6-a3bc-bc164d34c776 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:48.334045 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:48.359834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f86c-968b-43fc-813e-89194a79d87e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:48.432622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40111193-26a2-42c0-9af7-84c7d083e67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:48.435345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111193-26a2-42c0-9af7-84c7d083e67b None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:48.437278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111193-26a2-42c0-9af7-84c7d083e67b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:48.437736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40111193-26a2-42c0-9af7-84c7d083e67b None admin] PATH_INFO: `/v3/projects/17a73495723046079447956b439eb150/users/780b62fec93b4ec58e2b4cb92e229771/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:48.444116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40111193-26a2-42c0-9af7-84c7d083e67b None admin] RBAC: Authorizing `identity:create_grant(project_id=17a73495723046079447956b439eb150, user_id=780b62fec93b4ec58e2b4cb92e229771, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:48.445834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40111193-26a2-42c0-9af7-84c7d083e67b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:48.484307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 73/139] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:04:48 2022] PUT /identity/v3/projects/17a73495723046079447956b439eb150/users/780b62fec93b4ec58e2b4cb92e229771/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:04:48.502712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc9fa7-add0-4fe6-a3bc-bc164d34c776 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:48.549433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac61b7-c7a5-463e-918e-f99903647e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:48.550545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac61b7-c7a5-463e-918e-f99903647e3d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:48.550995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac61b7-c7a5-463e-918e-f99903647e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:48.551404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac61b7-c7a5-463e-918e-f99903647e3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:48.552835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac61b7-c7a5-463e-918e-f99903647e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:48.554498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac61b7-c7a5-463e-918e-f99903647e3d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:48.556607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 67/140] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:48.562135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40111193-26a2-42c0-9af7-84c7d083e67b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:48.727366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40111193-26a2-42c0-9af7-84c7d083e67b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:48.770056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c76aa2-7739-41e7-9b70-79b5f7516740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:48.771008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76aa2-7739-41e7-9b70-79b5f7516740 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:48.771448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76aa2-7739-41e7-9b70-79b5f7516740 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:48.771858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c76aa2-7739-41e7-9b70-79b5f7516740 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:48.772731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c76aa2-7739-41e7-9b70-79b5f7516740 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:48.774392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c76aa2-7739-41e7-9b70-79b5f7516740 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:48.792833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:48.816582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 74/141] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:04:48 2022] POST /identity/v3/users => generated 283 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:49.376127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbade-d3d6-40f1-a5fe-ea2900621459 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:49.376640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbade-d3d6-40f1-a5fe-ea2900621459 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:49.376712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbbade-d3d6-40f1-a5fe-ea2900621459 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:49.378259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 68/142] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:49.384413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cadc14-f5e7-4aaf-a249-9231e6d0e86d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:49.384707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cadc14-f5e7-4aaf-a249-9231e6d0e86d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:49.384972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cadc14-f5e7-4aaf-a249-9231e6d0e86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:49.406751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-28cadc14-f5e7-4aaf-a249-9231e6d0e86d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:49.407959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-28cadc14-f5e7-4aaf-a249-9231e6d0e86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:49.496547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 75/143] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:49.544149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbbade-d3d6-40f1-a5fe-ea2900621459 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:49.642934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:49.671196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbbade-d3d6-40f1-a5fe-ea2900621459 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:49.713138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb418e-2635-4c4e-aad1-0d5c81f20efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:49.714100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb418e-2635-4c4e-aad1-0d5c81f20efb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:49.714401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb418e-2635-4c4e-aad1-0d5c81f20efb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:49.714678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb418e-2635-4c4e-aad1-0d5c81f20efb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:49.735304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb418e-2635-4c4e-aad1-0d5c81f20efb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6feb418e-2635-4c4e-aad1-0d5c81f20efb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:49.743874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:49.744811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:49.745567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 69/144] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:04:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:49.751347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cadc14-f5e7-4aaf-a249-9231e6d0e86d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:49.867649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cadc14-f5e7-4aaf-a249-9231e6d0e86d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:49.894162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:49.908658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:49.909348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:49.909626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:49.909926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:49.910683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:49.910988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:49.912404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:49.922809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 76/145] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:04:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:49.928658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb418e-2635-4c4e-aad1-0d5c81f20efb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:50.053416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb418e-2635-4c4e-aad1-0d5c81f20efb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:50.096340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b17ae-dd38-4b29-b053-01e0b51c8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:50.097245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b17ae-dd38-4b29-b053-01e0b51c8464 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:50.097523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b17ae-dd38-4b29-b053-01e0b51c8464 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:50.097826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b17ae-dd38-4b29-b053-01e0b51c8464 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:50.099053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b17ae-dd38-4b29-b053-01e0b51c8464 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:50.100527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b17ae-dd38-4b29-b053-01e0b51c8464 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:50.102224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 70/146] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:49 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:50.108014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:50.236833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd34e42-e4b2-4e81-9cf6-e865cf87d07e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:50.258960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcd2aa-6483-4716-85cf-03ba0a09ad72 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:50.259503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcd2aa-6483-4716-85cf-03ba0a09ad72 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:50.259926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fcd2aa-6483-4716-85cf-03ba0a09ad72 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:50.261497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 71/147] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:04:50.267088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ca84a-6d54-4579-a2af-bf2d9e53c14a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:50.267607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ca84a-6d54-4579-a2af-bf2d9e53c14a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:50.268092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ca84a-6d54-4579-a2af-bf2d9e53c14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:50.274285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:50.299814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d29ca84a-6d54-4579-a2af-bf2d9e53c14a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:50.301579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d29ca84a-6d54-4579-a2af-bf2d9e53c14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:50.311489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3b668-a7b8-4590-a87a-503aaca9f9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:50.312430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b668-a7b8-4590-a87a-503aaca9f9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:50.312858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b668-a7b8-4590-a87a-503aaca9f9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:50.313263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3b668-a7b8-4590-a87a-503aaca9f9f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:50.422155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:50.445005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3b668-a7b8-4590-a87a-503aaca9f9f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:50.446573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3b668-a7b8-4590-a87a-503aaca9f9f6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:50.509882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 72/148] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:50.547061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ca84a-6d54-4579-a2af-bf2d9e53c14a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:50.604244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 77/149] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:50.610459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3b668-a7b8-4590-a87a-503aaca9f9f6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:50.755325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ca84a-6d54-4579-a2af-bf2d9e53c14a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:50.771887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:50.796352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3b668-a7b8-4590-a87a-503aaca9f9f6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:50.847039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e6ce2-cb1b-4bad-974b-dfe6071ab724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:50.847872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6ce2-cb1b-4bad-974b-dfe6071ab724 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:50.848164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6ce2-cb1b-4bad-974b-dfe6071ab724 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:50.848449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e6ce2-cb1b-4bad-974b-dfe6071ab724 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:50.849447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf0748-bfd7-498e-9e0f-baaa4aa87d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:50.850532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf0748-bfd7-498e-9e0f-baaa4aa87d09 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:50.851000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf0748-bfd7-498e-9e0f-baaa4aa87d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:50.851457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf0748-bfd7-498e-9e0f-baaa4aa87d09 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:50.855451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:50.857178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf0748-bfd7-498e-9e0f-baaa4aa87d09 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:50.858908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf0748-bfd7-498e-9e0f-baaa4aa87d09 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:50.860954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 78/150] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:50 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:50.863262 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d3999-71d5-4c5f-b829-f1ca7c3bb1f3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:50.863756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d3999-71d5-4c5f-b829-f1ca7c3bb1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:50.864203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d3999-71d5-4c5f-b829-f1ca7c3bb1f3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:50.865626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 79/151] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:50.869599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d3999-71d5-4c5f-b829-f1ca7c3bb1f3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:50.878675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e6ce2-cb1b-4bad-974b-dfe6071ab724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-827e6ce2-cb1b-4bad-974b-dfe6071ab724 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:50.890110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:50.891073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:04:50.891828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:50.896952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 73/152] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:04:50.898507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab5a1-14de-4b3b-8f73-84d5a542bc29 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:50.902511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab5a1-14de-4b3b-8f73-84d5a542bc29 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:50.902965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab5a1-14de-4b3b-8f73-84d5a542bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:50.953337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-449ab5a1-14de-4b3b-8f73-84d5a542bc29 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:50.955042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-449ab5a1-14de-4b3b-8f73-84d5a542bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:51.098831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d3999-71d5-4c5f-b829-f1ca7c3bb1f3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:51.121313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 74/153] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:51.122461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ab5a1-14de-4b3b-8f73-84d5a542bc29 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:51.194427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:51.195385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:51.195385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:51.195673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:51.196632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] RBAC: Adding query filter params (role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb, user.id=780b62fec93b4ec58e2b4cb92e229771) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:51.196911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:51.198511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:51.216267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 80/154] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:50 2022] GET /identity/v3/role_assignments?user.id=780b62fec93b4ec58e2b4cb92e229771&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 608 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:51.217046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:51.251686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ab5a1-14de-4b3b-8f73-84d5a542bc29 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:51.266035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:51.282199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:51.302889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:51.303984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:51.304463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:51.304932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:51.306048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:51.308392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:51.310171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:51.330079 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 75/155] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:51.338864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:51.436689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdabee9-1eb3-47e3-9034-42ce76ae358f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:51.494296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e828-81f3-4425-9856-23cd47dc9eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:51.496255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e828-81f3-4425-9856-23cd47dc9eab None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:51.496760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e828-81f3-4425-9856-23cd47dc9eab None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:51.497250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e828-81f3-4425-9856-23cd47dc9eab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:51.516958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1e828-81f3-4425-9856-23cd47dc9eab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-bca1e828-81f3-4425-9856-23cd47dc9eab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:51.530747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:51.531833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:51.532687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:51.535639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 81/156] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:51.552600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e828-81f3-4425-9856-23cd47dc9eab None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:51.558305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda8db8-9b78-41a8-b106-0ea0ae1cbd1d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:51.612955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2686934-1cb8-4e68-8366-19f7667e91a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:51.613883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686934-1cb8-4e68-8366-19f7667e91a6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:51.614172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686934-1cb8-4e68-8366-19f7667e91a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:51.614453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2686934-1cb8-4e68-8366-19f7667e91a6 None admin] PATH_INFO: `/v3/users/0d26578e1ae649f1ac2c3902b3cb25ad` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:51.615930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2686934-1cb8-4e68-8366-19f7667e91a6 None admin] RBAC: Authorizing `identity:get_user(user_id=0d26578e1ae649f1ac2c3902b3cb25ad)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:51.617173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2686934-1cb8-4e68-8366-19f7667e91a6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:51.619518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 76/157] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:51 2022] GET /identity/v3/users/0d26578e1ae649f1ac2c3902b3cb25ad => generated 289 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:51.625244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2686934-1cb8-4e68-8366-19f7667e91a6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:51.702109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:51.705455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:51.741988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e828-81f3-4425-9856-23cd47dc9eab None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:51.766143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2686934-1cb8-4e68-8366-19f7667e91a6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:51.796206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:51.797641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:51.798175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:51.798647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:51.799596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:51.800127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:51.801764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:51.816271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 82/158] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:51.820320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605fd1-11e8-4f44-9a0c-073101c0ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:51.821267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605fd1-11e8-4f44-9a0c-073101c0ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:51.821558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605fd1-11e8-4f44-9a0c-073101c0ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:51.821879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b605fd1-11e8-4f44-9a0c-073101c0ec04 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:51.825087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b605fd1-11e8-4f44-9a0c-073101c0ec04 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:51.826773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b605fd1-11e8-4f44-9a0c-073101c0ec04 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:51.828808 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 77/159] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:51 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:51.831369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:51.840623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605fd1-11e8-4f44-9a0c-073101c0ec04 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:52.012807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b605fd1-11e8-4f44-9a0c-073101c0ec04 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:52.031459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a2d61-d305-46ab-b3a9-d3eae061d167 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:52.054031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:52.075232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88931cb-c3f1-487e-b36f-cdd4567a7ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:52.076817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88931cb-c3f1-487e-b36f-cdd4567a7ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.078264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e8974-320b-4e57-aa63-996d324db4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:52.078830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88931cb-c3f1-487e-b36f-cdd4567a7ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.079308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88931cb-c3f1-487e-b36f-cdd4567a7ee1 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.080507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e8974-320b-4e57-aa63-996d324db4de None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.081743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e8974-320b-4e57-aa63-996d324db4de None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.082762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88931cb-c3f1-487e-b36f-cdd4567a7ee1 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:52.083281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e8974-320b-4e57-aa63-996d324db4de None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/users/0d26578e1ae649f1ac2c3902b3cb25ad/roles/6e98f7101fb9430a87421182e7b6f4a8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.086086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88931cb-c3f1-487e-b36f-cdd4567a7ee1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:52.087812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e8974-320b-4e57-aa63-996d324db4de None admin] RBAC: Authorizing `identity:create_grant(project_id=43134d25fcbf4ee6a4e0b977b35eddeb, user_id=0d26578e1ae649f1ac2c3902b3cb25ad, role_id=6e98f7101fb9430a87421182e7b6f4a8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:52.090852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 83/160] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:51 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:52.092732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e8974-320b-4e57-aa63-996d324db4de None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:52.096776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea77ba6-1643-4f8d-beef-3cdd0c00ddf9 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.097291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea77ba6-1643-4f8d-beef-3cdd0c00ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.098601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea77ba6-1643-4f8d-beef-3cdd0c00ddf9 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.100196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 84/161] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:52.102069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e0df6-57e1-4920-841d-f294f04c0efd None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.102543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e0df6-57e1-4920-841d-f294f04c0efd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.102990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e0df6-57e1-4920-841d-f294f04c0efd None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.105214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 85/162] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:52.106188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e0df6-57e1-4920-841d-f294f04c0efd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:52.115835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 78/163] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:04:51 2022] PUT /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/users/0d26578e1ae649f1ac2c3902b3cb25ad/roles/6e98f7101fb9430a87421182e7b6f4a8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:04:52.117370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23828e-0d83-43ae-b238-9dca8ecfdf41 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.117858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23828e-0d83-43ae-b238-9dca8ecfdf41 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.121608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23828e-0d83-43ae-b238-9dca8ecfdf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.130040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:52.162736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5d23828e-0d83-43ae-b238-9dca8ecfdf41 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:52.164099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5d23828e-0d83-43ae-b238-9dca8ecfdf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:52.304561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e0df6-57e1-4920-841d-f294f04c0efd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:52.324353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 79/164] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:52.325771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5838-13a7-4687-8170-55f3bd8bef6c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.326130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5838-13a7-4687-8170-55f3bd8bef6c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.326416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5838-13a7-4687-8170-55f3bd8bef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.351427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea83623-15a5-4b6b-977a-4cfa8935296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:52.351919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea83623-15a5-4b6b-977a-4cfa8935296c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.352204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea83623-15a5-4b6b-977a-4cfa8935296c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.352485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea83623-15a5-4b6b-977a-4cfa8935296c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.354534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5c0a5838-13a7-4687-8170-55f3bd8bef6c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:52.356035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5c0a5838-13a7-4687-8170-55f3bd8bef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:52.449225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 80/165] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:52.450181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a5838-13a7-4687-8170-55f3bd8bef6c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:52.509043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:52.511827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea83623-15a5-4b6b-977a-4cfa8935296c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:52.513374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea83623-15a5-4b6b-977a-4cfa8935296c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:52.525934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:52.580538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a5838-13a7-4687-8170-55f3bd8bef6c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:52.622223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478915a-63d7-47fd-b280-cafa63ee22c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:52.623037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478915a-63d7-47fd-b280-cafa63ee22c9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.623322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478915a-63d7-47fd-b280-cafa63ee22c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.623599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478915a-63d7-47fd-b280-cafa63ee22c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.646839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478915a-63d7-47fd-b280-cafa63ee22c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a478915a-63d7-47fd-b280-cafa63ee22c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:52.656405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:52.657311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:52.658610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:52.661386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 81/166] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:04:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:52.662595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478915a-63d7-47fd-b280-cafa63ee22c9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:52.781629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478915a-63d7-47fd-b280-cafa63ee22c9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:52.783777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:52.826085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a37c74-3d40-4933-8c1a-6edac2d541c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:52.827383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a37c74-3d40-4933-8c1a-6edac2d541c5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:52.827935 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a37c74-3d40-4933-8c1a-6edac2d541c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:52.828417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a37c74-3d40-4933-8c1a-6edac2d541c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:52.840539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 86/167] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:52.842155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea83623-15a5-4b6b-977a-4cfa8935296c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:52.845082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a37c74-3d40-4933-8c1a-6edac2d541c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c2a37c74-3d40-4933-8c1a-6edac2d541c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:52.865720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:52.866662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 82/168] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:52.867896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a37c74-3d40-4933-8c1a-6edac2d541c5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:53.037588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a37c74-3d40-4933-8c1a-6edac2d541c5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:53.068912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea83623-15a5-4b6b-977a-4cfa8935296c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:53.078912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b4b1c-65e8-445d-9b0f-ab2f58b1d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:53.080105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4b1c-65e8-445d-9b0f-ab2f58b1d0dd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.080546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4b1c-65e8-445d-9b0f-ab2f58b1d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.080978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b4b1c-65e8-445d-9b0f-ab2f58b1d0dd None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.082375 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b4b1c-65e8-445d-9b0f-ab2f58b1d0dd None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:53.084020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b4b1c-65e8-445d-9b0f-ab2f58b1d0dd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:53.085956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 83/169] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:52 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:53.086895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b4b1c-65e8-445d-9b0f-ab2f58b1d0dd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:53.118129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:53.123597 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:53.134417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:53.135131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.135451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.136167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.136485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:53.136797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:53.138264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:53.156087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 87/170] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:53.157037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:53.206053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b4b1c-65e8-445d-9b0f-ab2f58b1d0dd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:53.242466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f081bd91-e9e9-4fa1-b725-bc2e63a21cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:53.243138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081bd91-e9e9-4fa1-b725-bc2e63a21cdb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.243727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081bd91-e9e9-4fa1-b725-bc2e63a21cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.243999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081bd91-e9e9-4fa1-b725-bc2e63a21cdb None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.244787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081bd91-e9e9-4fa1-b725-bc2e63a21cdb None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:53.245205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081bd91-e9e9-4fa1-b725-bc2e63a21cdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:53.246891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f081bd91-e9e9-4fa1-b725-bc2e63a21cdb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:53.256217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 84/171] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:53.257612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96722118-a7d9-4982-802c-06b85907ab08 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.266228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96722118-a7d9-4982-802c-06b85907ab08 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.266524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96722118-a7d9-4982-802c-06b85907ab08 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.268001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 85/172] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:53.268887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96722118-a7d9-4982-802c-06b85907ab08 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:53.301529 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0750d77d-48e5-4d32-851c-765df2553395 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:53.365213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:53.365918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.366204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.366481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.367326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] RBAC: Adding query filter params (role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb, user.id=780b62fec93b4ec58e2b4cb92e229771) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:53.367642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:53.369489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:53.380178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 88/173] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:53 2022] GET /identity/v3/role_assignments?user.id=780b62fec93b4ec58e2b4cb92e229771&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:53.384098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:53.465968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:53.488131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:53.516660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96722118-a7d9-4982-802c-06b85907ab08 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:53.597518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd010f-672e-47c8-ae56-aeca7f30551a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:53.631185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31992e-fcae-4174-830f-8dadf8a7d46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:53.631863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31992e-fcae-4174-830f-8dadf8a7d46f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.631863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31992e-fcae-4174-830f-8dadf8a7d46f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.632245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31992e-fcae-4174-830f-8dadf8a7d46f None admin] PATH_INFO: `/v3/users/86f9463b664941f0b5c63b336b679b58` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.637080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31992e-fcae-4174-830f-8dadf8a7d46f None admin] RBAC: Authorizing `identity:get_user(user_id=86f9463b664941f0b5c63b336b679b58)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:53.638629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31992e-fcae-4174-830f-8dadf8a7d46f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:53.640409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 86/174] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:53 2022] GET /identity/v3/users/86f9463b664941f0b5c63b336b679b58 => generated 247 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:53.643768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715ce2c-74d8-419f-94af-37b671b18fc7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.643768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715ce2c-74d8-419f-94af-37b671b18fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.643768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8715ce2c-74d8-419f-94af-37b671b18fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.657714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a19bc1-8d85-425f-aa4e-e2d1c53ff238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:53.657714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a19bc1-8d85-425f-aa4e-e2d1c53ff238 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.657714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a19bc1-8d85-425f-aa4e-e2d1c53ff238 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.657714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a19bc1-8d85-425f-aa4e-e2d1c53ff238 None admin] PATH_INFO: `/v3/users/780b62fec93b4ec58e2b4cb92e229771` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.661259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a19bc1-8d85-425f-aa4e-e2d1c53ff238 None admin] RBAC: Authorizing `identity:get_user(user_id=780b62fec93b4ec58e2b4cb92e229771)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:53.662809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a19bc1-8d85-425f-aa4e-e2d1c53ff238 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:53.664560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 89/175] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:53 2022] GET /identity/v3/users/780b62fec93b4ec58e2b4cb92e229771 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:53.665335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a19bc1-8d85-425f-aa4e-e2d1c53ff238 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:53.675678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8715ce2c-74d8-419f-94af-37b671b18fc7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:53.677269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8715ce2c-74d8-419f-94af-37b671b18fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:53.786309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:53.790747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a19bc1-8d85-425f-aa4e-e2d1c53ff238 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:53.823193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 87/176] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:53.827319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715ce2c-74d8-419f-94af-37b671b18fc7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:53.847048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018c514-5796-422c-918b-ceefcb8704d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:53.847719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c514-5796-422c-918b-ceefcb8704d0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:53.848011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c514-5796-422c-918b-ceefcb8704d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:53.848289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c514-5796-422c-918b-ceefcb8704d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:53.950294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018c514-5796-422c-918b-ceefcb8704d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:53.951746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018c514-5796-422c-918b-ceefcb8704d0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:53.987477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8715ce2c-74d8-419f-94af-37b671b18fc7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:53.990415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.026106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066862a-aa7c-4a38-bbbc-1e02d2cbf1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.026819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066862a-aa7c-4a38-bbbc-1e02d2cbf1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.027106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066862a-aa7c-4a38-bbbc-1e02d2cbf1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.027386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066862a-aa7c-4a38-bbbc-1e02d2cbf1d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.075379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.084277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 90/177] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:54.085755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9186c2-ec8c-484d-af34-8b32faa8774f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.086070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9186c2-ec8c-484d-af34-8b32faa8774f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.086419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9186c2-ec8c-484d-af34-8b32faa8774f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.087963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 91/178] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:54.092063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9186c2-ec8c-484d-af34-8b32faa8774f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.101148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066862a-aa7c-4a38-bbbc-1e02d2cbf1d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:54.102673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066862a-aa7c-4a38-bbbc-1e02d2cbf1d2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:54.192194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.200143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9186c2-ec8c-484d-af34-8b32faa8774f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:54.207932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 88/179] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:54.207932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da169d1-cebf-4ac4-bccd-dada3b0e0cf1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.207932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da169d1-cebf-4ac4-bccd-dada3b0e0cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.207932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da169d1-cebf-4ac4-bccd-dada3b0e0cf1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.207932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 89/180] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:54.208311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da169d1-cebf-4ac4-bccd-dada3b0e0cf1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.241872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270b811-b215-4e66-93c4-0dba79a76d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.242712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270b811-b215-4e66-93c4-0dba79a76d42 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.243056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270b811-b215-4e66-93c4-0dba79a76d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.243641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270b811-b215-4e66-93c4-0dba79a76d42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.254664 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270b811-b215-4e66-93c4-0dba79a76d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0270b811-b215-4e66-93c4-0dba79a76d42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:54.262633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:54.263721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:54.264559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 92/181] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:54.265218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6d822-8c9e-44f9-a31a-a5191601a078 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.265333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6d822-8c9e-44f9-a31a-a5191601a078 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.265614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6d822-8c9e-44f9-a31a-a5191601a078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.284264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d8c6d822-8c9e-44f9-a31a-a5191601a078 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:54.285593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d8c6d822-8c9e-44f9-a31a-a5191601a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:54.324766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da169d1-cebf-4ac4-bccd-dada3b0e0cf1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:54.345469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.359699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 93/182] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:54.360329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6d822-8c9e-44f9-a31a-a5191601a078 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.361963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f55478-6bd0-4b4c-bfe7-8d75256bbe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.364186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f55478-6bd0-4b4c-bfe7-8d75256bbe07 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.364676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f55478-6bd0-4b4c-bfe7-8d75256bbe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.365154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f55478-6bd0-4b4c-bfe7-8d75256bbe07 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.366680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f55478-6bd0-4b4c-bfe7-8d75256bbe07 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:54.369129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f55478-6bd0-4b4c-bfe7-8d75256bbe07 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:54.371655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 90/183] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:54.373905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375fe2-fa60-4934-ac40-11670caa1478 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.374430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375fe2-fa60-4934-ac40-11670caa1478 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.375061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68375fe2-fa60-4934-ac40-11670caa1478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.395328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-68375fe2-fa60-4934-ac40-11670caa1478 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:54.396497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-68375fe2-fa60-4934-ac40-11670caa1478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:54.452753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.481732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 91/184] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:54.481732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68375fe2-fa60-4934-ac40-11670caa1478 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.491740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6d822-8c9e-44f9-a31a-a5191601a078 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:54.530398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3852d-05ea-4027-9355-1d0c8d864000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.531278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3852d-05ea-4027-9355-1d0c8d864000 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.531564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3852d-05ea-4027-9355-1d0c8d864000 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.531844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3852d-05ea-4027-9355-1d0c8d864000 None admin] PATH_INFO: `/v3/projects/17a73495723046079447956b439eb150` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.533270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3852d-05ea-4027-9355-1d0c8d864000 None admin] RBAC: Authorizing `identity:get_project(project_id=17a73495723046079447956b439eb150)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:54.534859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d3852d-05ea-4027-9355-1d0c8d864000 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:54.537178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 94/185] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/projects/17a73495723046079447956b439eb150 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:54.537998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3852d-05ea-4027-9355-1d0c8d864000 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.592263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68375fe2-fa60-4934-ac40-11670caa1478 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:54.600903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.628018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.628692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.629022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.629298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.630117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:54.630428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:54.631814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:54.643643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 92/186] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:54.644489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.660710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d3852d-05ea-4027-9355-1d0c8d864000 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:54.701688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.705431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.706134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.706411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.706683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.707525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb, user.id=86f9463b664941f0b5c63b336b679b58) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:54.707841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:54.709307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:54.721688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 95/187] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/role_assignments?user.id=86f9463b664941f0b5c63b336b679b58&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=5c672a1b2979444383c587a215adeb35 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:54.721688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.767311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a9e3d-b5b1-4323-9214-a29260c7f275 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:54.811651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363bfbc-a082-423b-8f7a-26a21007d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.812485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363bfbc-a082-423b-8f7a-26a21007d855 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.812790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363bfbc-a082-423b-8f7a-26a21007d855 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.813067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b363bfbc-a082-423b-8f7a-26a21007d855 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.814599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363bfbc-a082-423b-8f7a-26a21007d855 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:54.816309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b363bfbc-a082-423b-8f7a-26a21007d855 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:54.818358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 93/188] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:54.819020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363bfbc-a082-423b-8f7a-26a21007d855 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.854157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93635c94-273c-4e1b-99c4-229e17c3b76d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:54.883072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.900244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062ae77-2645-4fc3-a504-8d559828dae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.901842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062ae77-2645-4fc3-a504-8d559828dae9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.901973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062ae77-2645-4fc3-a504-8d559828dae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.902191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062ae77-2645-4fc3-a504-8d559828dae9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.917089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062ae77-2645-4fc3-a504-8d559828dae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8062ae77-2645-4fc3-a504-8d559828dae9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:54.925450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:54.926676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 96/189] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:54.927552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062ae77-2645-4fc3-a504-8d559828dae9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:54.943117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b363bfbc-a082-423b-8f7a-26a21007d855 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:54.966692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:54.982039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:54.982776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:54.982988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:54.983253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:54.984107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] RBAC: Adding query filter params (role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=17a73495723046079447956b439eb150, user.id=780b62fec93b4ec58e2b4cb92e229771) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:54.984440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:54.985993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:55.000504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 94/190] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/role_assignments?user.id=780b62fec93b4ec58e2b4cb92e229771&scope.project.id=17a73495723046079447956b439eb150&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:55.000504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:55.079593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062ae77-2645-4fc3-a504-8d559828dae9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:55.113857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68cd1a-8b1e-4cdc-a143-06bd2bb726f9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:55.124081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fccf0-936f-4e98-a1fa-847e7484f23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:55.125544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fccf0-936f-4e98-a1fa-847e7484f23b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.125865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fccf0-936f-4e98-a1fa-847e7484f23b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.126147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fccf0-936f-4e98-a1fa-847e7484f23b None admin] PATH_INFO: `/v3/users/fb79248689d5403d8af3fce64dbdebbc` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.143815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fccf0-936f-4e98-a1fa-847e7484f23b None admin] RBAC: Authorizing `identity:get_user(user_id=fb79248689d5403d8af3fce64dbdebbc)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:55.145577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5fccf0-936f-4e98-a1fa-847e7484f23b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:55.148214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 97/191] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:54 2022] GET /identity/v3/users/fb79248689d5403d8af3fce64dbdebbc => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:55.148768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15be5-b8ab-4360-bcf4-ca05169ed5a9 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.149009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15be5-b8ab-4360-bcf4-ca05169ed5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.149288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b15be5-b8ab-4360-bcf4-ca05169ed5a9 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.150807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 98/192] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:55.152271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b15be5-b8ab-4360-bcf4-ca05169ed5a9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:55.159397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b66f0-0eef-43f5-832e-e4521dca9ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:55.160090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b66f0-0eef-43f5-832e-e4521dca9ca3 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.160382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b66f0-0eef-43f5-832e-e4521dca9ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.160657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b66f0-0eef-43f5-832e-e4521dca9ca3 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.161377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b66f0-0eef-43f5-832e-e4521dca9ca3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:55.162921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b66f0-0eef-43f5-832e-e4521dca9ca3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:55.184640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 95/193] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:04:54 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:55.186444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b66f0-0eef-43f5-832e-e4521dca9ca3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:55.211452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:55.237750 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:55.313957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b15be5-b8ab-4360-bcf4-ca05169ed5a9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:55.329070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b66f0-0eef-43f5-832e-e4521dca9ca3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:55.367351 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27dd08-922b-4385-be5c-a6a2e45806fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:55.367589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499e4ef-5754-4937-8b76-03d2a4387488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:55.369438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27dd08-922b-4385-be5c-a6a2e45806fb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.369713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499e4ef-5754-4937-8b76-03d2a4387488 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.369853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27dd08-922b-4385-be5c-a6a2e45806fb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.370651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499e4ef-5754-4937-8b76-03d2a4387488 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.370765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27dd08-922b-4385-be5c-a6a2e45806fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.370890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499e4ef-5754-4937-8b76-03d2a4387488 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.370890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499e4ef-5754-4937-8b76-03d2a4387488 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:55.370890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499e4ef-5754-4937-8b76-03d2a4387488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:55.371726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499e4ef-5754-4937-8b76-03d2a4387488 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:55.383649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 99/194] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:04:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:55.385848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ab2e-437b-4850-a947-4e30ccc7410c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.386686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ab2e-437b-4850-a947-4e30ccc7410c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.387095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6ab2e-437b-4850-a947-4e30ccc7410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.409066 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1cd6ab2e-437b-4850-a947-4e30ccc7410c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:55.410310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1cd6ab2e-437b-4850-a947-4e30ccc7410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:55.456207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27dd08-922b-4385-be5c-a6a2e45806fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:55.458342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27dd08-922b-4385-be5c-a6a2e45806fb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:55.505898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:55.509071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 100/195] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:55.509730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ab2e-437b-4850-a947-4e30ccc7410c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:55.554764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:55.580181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 96/196] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:55.581503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27dd08-922b-4385-be5c-a6a2e45806fb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:55.650855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6ab2e-437b-4850-a947-4e30ccc7410c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:55.690726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa6500-e5d1-4365-a481-8780eb02143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:55.692006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6500-e5d1-4365-a481-8780eb02143c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.693064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6500-e5d1-4365-a481-8780eb02143c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.693556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6500-e5d1-4365-a481-8780eb02143c None admin] PATH_INFO: `/v3/users/fb79248689d5403d8af3fce64dbdebbc` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.695677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa6500-e5d1-4365-a481-8780eb02143c None admin] RBAC: Authorizing `identity:get_user(user_id=fb79248689d5403d8af3fce64dbdebbc)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:55.699048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa6500-e5d1-4365-a481-8780eb02143c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:55.701756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 101/197] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:55 2022] GET /identity/v3/users/fb79248689d5403d8af3fce64dbdebbc => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:55.703095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa6500-e5d1-4365-a481-8780eb02143c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:55.706477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27dd08-922b-4385-be5c-a6a2e45806fb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:55.746029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea9b50-5159-44c0-ba2d-1f02342b821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:55.747303 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea9b50-5159-44c0-ba2d-1f02342b821c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.747693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea9b50-5159-44c0-ba2d-1f02342b821c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.748094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea9b50-5159-44c0-ba2d-1f02342b821c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.757535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:55.764366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea9b50-5159-44c0-ba2d-1f02342b821c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8dea9b50-5159-44c0-ba2d-1f02342b821c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:55.774710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:55.775643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:55.776810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 97/198] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:55.777474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea9b50-5159-44c0-ba2d-1f02342b821c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:55.832728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:55.836678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa6500-e5d1-4365-a481-8780eb02143c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:55.882479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59165881-a9fa-4f96-91c3-0c8531c53a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:55.883371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165881-a9fa-4f96-91c3-0c8531c53a43 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.883649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165881-a9fa-4f96-91c3-0c8531c53a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.883922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165881-a9fa-4f96-91c3-0c8531c53a43 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:55.885247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59165881-a9fa-4f96-91c3-0c8531c53a43 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:55.887315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59165881-a9fa-4f96-91c3-0c8531c53a43 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:55.889763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 102/199] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:55 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:55.890870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59165881-a9fa-4f96-91c3-0c8531c53a43 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:55.915679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea9b50-5159-44c0-ba2d-1f02342b821c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:55.953782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2dfe0-994c-4eb6-b679-90c442418696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:55.954489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2dfe0-994c-4eb6-b679-90c442418696 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:55.954838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2dfe0-994c-4eb6-b679-90c442418696 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:55.955124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2dfe0-994c-4eb6-b679-90c442418696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:56.005760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59165881-a9fa-4f96-91c3-0c8531c53a43 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:56.039265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:56.044221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:56.044912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:56.045192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:56.045290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2dfe0-994c-4eb6-b679-90c442418696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:56.045479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:56.046323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:56.046675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:56.046845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2dfe0-994c-4eb6-b679-90c442418696 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:56.048070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:56.059318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 103/200] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:56.060213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:56.093985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:56.171144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269b527-79a6-4adf-9ee2-772e0acadc7a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:56.181450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 98/201] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:56.182459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2dfe0-994c-4eb6-b679-90c442418696 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:56.211117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:56.212084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:56.212533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:56.213095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:56.214465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=5c8b586142aa493484ff06e0a70778cf, user.id=fb79248689d5403d8af3fce64dbdebbc) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:56.214962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:56.216614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:56.222773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 104/202] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:56 2022] GET /identity/v3/role_assignments?user.id=fb79248689d5403d8af3fce64dbdebbc&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=5c672a1b2979444383c587a215adeb35 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:56.223883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:56.307267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:56.312608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2dfe0-994c-4eb6-b679-90c442418696 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:56.340855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:56.350739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e59d-a29e-4e80-945a-8a45eedd06f1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:56.370559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482814de-bd32-453a-a6f1-6392e7716603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:56.371468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482814de-bd32-453a-a6f1-6392e7716603 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:56.371754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482814de-bd32-453a-a6f1-6392e7716603 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:56.372033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482814de-bd32-453a-a6f1-6392e7716603 None admin] PATH_INFO: `/v3/users/0d26578e1ae649f1ac2c3902b3cb25ad` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:56.373381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482814de-bd32-453a-a6f1-6392e7716603 None admin] RBAC: Authorizing `identity:get_user(user_id=0d26578e1ae649f1ac2c3902b3cb25ad)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:56.374913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482814de-bd32-453a-a6f1-6392e7716603 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:56.376587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 99/203] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:56 2022] GET /identity/v3/users/0d26578e1ae649f1ac2c3902b3cb25ad => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:56.382532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482814de-bd32-453a-a6f1-6392e7716603 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:56.394446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554e73c-21ae-428f-b398-ae9db1f96b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:56.395705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554e73c-21ae-428f-b398-ae9db1f96b1f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:56.396191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554e73c-21ae-428f-b398-ae9db1f96b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:56.396785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554e73c-21ae-428f-b398-ae9db1f96b1f None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:56.398312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554e73c-21ae-428f-b398-ae9db1f96b1f None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:56.400120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554e73c-21ae-428f-b398-ae9db1f96b1f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:56.402433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 105/204] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:56 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:56.410837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554e73c-21ae-428f-b398-ae9db1f96b1f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:56.524112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482814de-bd32-453a-a6f1-6392e7716603 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:56.548324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554e73c-21ae-428f-b398-ae9db1f96b1f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:56.565146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324c527-8d4a-4ea2-839c-55cb65626c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:56.566313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c527-8d4a-4ea2-839c-55cb65626c57 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:56.566799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c527-8d4a-4ea2-839c-55cb65626c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:56.567291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324c527-8d4a-4ea2-839c-55cb65626c57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:56.596787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:56.599009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:56.599521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:56.602886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:56.605713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] RBAC: Adding query filter params (role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=5c8b586142aa493484ff06e0a70778cf, user.id=fb79248689d5403d8af3fce64dbdebbc) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:56.606435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:56.609326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:56.621015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 106/205] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:56 2022] GET /identity/v3/role_assignments?user.id=fb79248689d5403d8af3fce64dbdebbc&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:56.628176 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:56.678482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324c527-8d4a-4ea2-839c-55cb65626c57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:56.680053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324c527-8d4a-4ea2-839c-55cb65626c57 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:56.804843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 100/206] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:04:56.810745 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:56.848586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:56.933140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20904eed-1839-4887-916d-ff37d66f3130 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:56.973241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e7605-1c0b-4574-9166-ac50905e01dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:56.974153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e7605-1c0b-4574-9166-ac50905e01dd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:56.974435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e7605-1c0b-4574-9166-ac50905e01dd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:56.974706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e7605-1c0b-4574-9166-ac50905e01dd None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:56.975904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e7605-1c0b-4574-9166-ac50905e01dd None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:56.977375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e7605-1c0b-4574-9166-ac50905e01dd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:56.979407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 107/207] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:56 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:56.985509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324c527-8d4a-4ea2-839c-55cb65626c57 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:57.071585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:57.125027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324c527-8d4a-4ea2-839c-55cb65626c57 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:57.184956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0311f56-ebca-4cde-86e0-8b743336010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:57.185646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0311f56-ebca-4cde-86e0-8b743336010f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:57.185956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0311f56-ebca-4cde-86e0-8b743336010f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:57.186235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0311f56-ebca-4cde-86e0-8b743336010f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:57.187088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0311f56-ebca-4cde-86e0-8b743336010f None admin] RBAC: Adding query filter params (role.id=6e98f7101fb9430a87421182e7b6f4a8, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb, user.id=0d26578e1ae649f1ac2c3902b3cb25ad) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:57.187403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0311f56-ebca-4cde-86e0-8b743336010f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:57.188881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0311f56-ebca-4cde-86e0-8b743336010f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:57.205340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 101/208] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:56 2022] GET /identity/v3/role_assignments?user.id=0d26578e1ae649f1ac2c3902b3cb25ad&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=6e98f7101fb9430a87421182e7b6f4a8 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:57.629716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5acc7-ab61-4d3a-b104-01f77de35117 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:57.630117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5acc7-ab61-4d3a-b104-01f77de35117 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:57.630294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5acc7-ab61-4d3a-b104-01f77de35117 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:57.635087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 108/209] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:57.641997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a03e3-569b-4f80-8bb6-3d20904df348 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:57.642511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a03e3-569b-4f80-8bb6-3d20904df348 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:57.642991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a03e3-569b-4f80-8bb6-3d20904df348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:57.658658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05aa49c-7fac-45b5-a035-bb39c0cac9c4 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:57.658955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05aa49c-7fac-45b5-a035-bb39c0cac9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:57.659235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05aa49c-7fac-45b5-a035-bb39c0cac9c4 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:57.660764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 109/210] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:57.666156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc859-2e08-4cc0-af5b-0b50ef841164 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:57.666491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc859-2e08-4cc0-af5b-0b50ef841164 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:57.666821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc859-2e08-4cc0-af5b-0b50ef841164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:57.673097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-475a03e3-569b-4f80-8bb6-3d20904df348 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:57.674325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-475a03e3-569b-4f80-8bb6-3d20904df348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:57.692269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b3cbc859-2e08-4cc0-af5b-0b50ef841164 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:57.693852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b3cbc859-2e08-4cc0-af5b-0b50ef841164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:57.777048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 102/211] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:57.800195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 110/212] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:57.807104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a03e3-569b-4f80-8bb6-3d20904df348 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:57.827097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbc859-2e08-4cc0-af5b-0b50ef841164 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:57.855719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:57.880704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:57.932276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a03e3-569b-4f80-8bb6-3d20904df348 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:57.945504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbc859-2e08-4cc0-af5b-0b50ef841164 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:57.988487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a52025-7e73-4ce2-8c6a-34df192eca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:57.989296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a52025-7e73-4ce2-8c6a-34df192eca9c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:57.989579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a52025-7e73-4ce2-8c6a-34df192eca9c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:57.989898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a52025-7e73-4ce2-8c6a-34df192eca9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:57.991127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a52025-7e73-4ce2-8c6a-34df192eca9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:57.992391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f20129-f0e9-471e-8b7e-d22e6a42ec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:57.992650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a52025-7e73-4ce2-8c6a-34df192eca9c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:57.993933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f20129-f0e9-471e-8b7e-d22e6a42ec21 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:57.994409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f20129-f0e9-471e-8b7e-d22e6a42ec21 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:57.994777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 103/213] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:04:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:57.995746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f20129-f0e9-471e-8b7e-d22e6a42ec21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:58.000803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a52025-7e73-4ce2-8c6a-34df192eca9c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:58.009522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f20129-f0e9-471e-8b7e-d22e6a42ec21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c2f20129-f0e9-471e-8b7e-d22e6a42ec21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:58.019916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:58.020813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:04:58.021626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:58.024147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 111/214] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:04:58.030524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f20129-f0e9-471e-8b7e-d22e6a42ec21 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:58.116690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a52025-7e73-4ce2-8c6a-34df192eca9c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:58.146595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:58.153726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:58.157586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960583c-7684-40a3-bafd-2918a891b039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:58.159092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960583c-7684-40a3-bafd-2918a891b039 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:58.159583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960583c-7684-40a3-bafd-2918a891b039 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:58.160097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960583c-7684-40a3-bafd-2918a891b039 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:58.161114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960583c-7684-40a3-bafd-2918a891b039 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:58.163117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960583c-7684-40a3-bafd-2918a891b039 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:58.171031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f20129-f0e9-471e-8b7e-d22e6a42ec21 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:58.191330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 104/215] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:04:57 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:04:58.224024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:58.225161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:58.225700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:58.226225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:58.227303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:58.227831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:58.229927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:58.241452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 112/216] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:04:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:58.251803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960583c-7684-40a3-bafd-2918a891b039 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:58.384575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960583c-7684-40a3-bafd-2918a891b039 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:58.423711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723d40b-3c4b-4e93-89f0-bfde1303e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:58.424939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723d40b-3c4b-4e93-89f0-bfde1303e020 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:58.425459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723d40b-3c4b-4e93-89f0-bfde1303e020 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:58.425959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723d40b-3c4b-4e93-89f0-bfde1303e020 None admin] PATH_INFO: `/v3/users/a4d1597246dc498b9adf86d0a8670054` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:58.444281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723d40b-3c4b-4e93-89f0-bfde1303e020 None admin] RBAC: Authorizing `identity:get_user(user_id=a4d1597246dc498b9adf86d0a8670054)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:58.446854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723d40b-3c4b-4e93-89f0-bfde1303e020 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:58.450232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 105/217] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:58 2022] GET /identity/v3/users/a4d1597246dc498b9adf86d0a8670054 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:58.456152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:58.573497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59463ccf-50c5-4d80-8545-85325843e6e0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:58.610883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d12da-6a41-4c77-b245-9801d191adcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:58.611894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d12da-6a41-4c77-b245-9801d191adcb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:58.612391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d12da-6a41-4c77-b245-9801d191adcb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:58.612882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d12da-6a41-4c77-b245-9801d191adcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:58.632954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:58.693435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d12da-6a41-4c77-b245-9801d191adcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:58.695343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d12da-6a41-4c77-b245-9801d191adcb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:58.734960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2f3d-ceab-4a16-87d0-4dfc6c81dce0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:58.735269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2f3d-ceab-4a16-87d0-4dfc6c81dce0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:58.735774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba2f3d-ceab-4a16-87d0-4dfc6c81dce0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:58.737506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 106/218] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:58.742858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e1b31-0fcf-4145-90cb-064215d1eca6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:58.743145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e1b31-0fcf-4145-90cb-064215d1eca6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:58.743802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e1b31-0fcf-4145-90cb-064215d1eca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:58.764084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0f2e1b31-0fcf-4145-90cb-064215d1eca6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:58.765411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0f2e1b31-0fcf-4145-90cb-064215d1eca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:58.797516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 113/219] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:04:58.803320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d12da-6a41-4c77-b245-9801d191adcb None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:58.841805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:58.857600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:58.862826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 107/220] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:04:58.886748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e1b31-0fcf-4145-90cb-064215d1eca6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:58.950437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d12da-6a41-4c77-b245-9801d191adcb None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:58.996628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f90d3-99cb-472f-bfb5-4483cd46c93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:58.997522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f90d3-99cb-472f-bfb5-4483cd46c93a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:58.997913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f90d3-99cb-472f-bfb5-4483cd46c93a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:58.998184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f90d3-99cb-472f-bfb5-4483cd46c93a None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.002550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f90d3-99cb-472f-bfb5-4483cd46c93a None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:59.004131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f90d3-99cb-472f-bfb5-4483cd46c93a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:59.006251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 114/221] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:58 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:59.006617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e1b31-0fcf-4145-90cb-064215d1eca6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:59.013968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f90d3-99cb-472f-bfb5-4483cd46c93a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:59.050194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bce72-bbb7-48cd-a2c0-1a807696c34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:59.052301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bce72-bbb7-48cd-a2c0-1a807696c34f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.052733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bce72-bbb7-48cd-a2c0-1a807696c34f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.053280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bce72-bbb7-48cd-a2c0-1a807696c34f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.067606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bce72-bbb7-48cd-a2c0-1a807696c34f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-979bce72-bbb7-48cd-a2c0-1a807696c34f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:59.076353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:59.077313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:04:59.078141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:04:59.080923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 108/222] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:04:59.090971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bce72-bbb7-48cd-a2c0-1a807696c34f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:59.146029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:59.230372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f90d3-99cb-472f-bfb5-4483cd46c93a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:59.252073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:59.255948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bce72-bbb7-48cd-a2c0-1a807696c34f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:59.274275 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e60b5-1733-4262-b085-57c8d861909e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:59.274942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e60b5-1733-4262-b085-57c8d861909e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.275221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e60b5-1733-4262-b085-57c8d861909e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.275498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e60b5-1733-4262-b085-57c8d861909e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.276443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e60b5-1733-4262-b085-57c8d861909e None admin] RBAC: Adding query filter params (role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=5c8b586142aa493484ff06e0a70778cf, user.id=a4d1597246dc498b9adf86d0a8670054) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:59.276694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e60b5-1733-4262-b085-57c8d861909e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:59.278338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e60b5-1733-4262-b085-57c8d861909e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:59.284471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 115/223] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:04:59 2022] GET /identity/v3/role_assignments?user.id=a4d1597246dc498b9adf86d0a8670054&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:59.293700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acb7d9-0f22-41e3-9d4c-1b2e7633ef47 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.293700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acb7d9-0f22-41e3-9d4c-1b2e7633ef47 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.293700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acb7d9-0f22-41e3-9d4c-1b2e7633ef47 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.293700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 116/224] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:04:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:04:59.294085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c89b-d6b4-4ae2-9980-01dfb736b5be None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.294085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c89b-d6b4-4ae2-9980-01dfb736b5be None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.294437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c09c89b-d6b4-4ae2-9980-01dfb736b5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.310414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:59.311513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.311999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.312461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8c09c89b-d6b4-4ae2-9980-01dfb736b5be None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:04:59.312689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.313668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8c09c89b-d6b4-4ae2-9980-01dfb736b5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:04:59.313914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:59.314417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:59.316058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:59.326569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 109/225] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:59.332572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:59.422419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 117/226] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:04:59.468578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09c89b-d6b4-4ae2-9980-01dfb736b5be None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:59.517060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:59.527872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0e3b4-1669-4a22-80d9-bd3a2974ae3b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:59.581108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29276a49-bab8-46c7-acd1-8faf948d717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:59.582047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29276a49-bab8-46c7-acd1-8faf948d717a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.582331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29276a49-bab8-46c7-acd1-8faf948d717a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.582605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29276a49-bab8-46c7-acd1-8faf948d717a None admin] PATH_INFO: `/v3/users/fb79248689d5403d8af3fce64dbdebbc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.586554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29276a49-bab8-46c7-acd1-8faf948d717a None admin] RBAC: Authorizing `identity:get_user(user_id=fb79248689d5403d8af3fce64dbdebbc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:59.588062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29276a49-bab8-46c7-acd1-8faf948d717a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:59.590274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 110/227] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:59 2022] GET /identity/v3/users/fb79248689d5403d8af3fce64dbdebbc => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:59.595917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29276a49-bab8-46c7-acd1-8faf948d717a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:59.605868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c09c89b-d6b4-4ae2-9980-01dfb736b5be None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:59.616092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:59.653588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0b930-0bf0-4f55-a643-d58ec82b7fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:59.654547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0b930-0bf0-4f55-a643-d58ec82b7fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.654982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0b930-0bf0-4f55-a643-d58ec82b7fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.655286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0b930-0bf0-4f55-a643-d58ec82b7fc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.667773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0b930-0bf0-4f55-a643-d58ec82b7fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-b7d0b930-0bf0-4f55-a643-d58ec82b7fc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:04:59.676355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:04:59.677295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:04:59.678124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 118/228] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:04:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:04:59.686662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0b930-0bf0-4f55-a643-d58ec82b7fc6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:59.750556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29276a49-bab8-46c7-acd1-8faf948d717a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:59.816299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e676fb1-eb03-43e1-ac06-87e046ad7f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:59.817094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676fb1-eb03-43e1-ac06-87e046ad7f27 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.817380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676fb1-eb03-43e1-ac06-87e046ad7f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.817667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e676fb1-eb03-43e1-ac06-87e046ad7f27 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.817789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0b930-0bf0-4f55-a643-d58ec82b7fc6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:59.818888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e676fb1-eb03-43e1-ac06-87e046ad7f27 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:59.820349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e676fb1-eb03-43e1-ac06-87e046ad7f27 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:59.822293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 111/229] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:04:59 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:04:59.827921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e676fb1-eb03-43e1-ac06-87e046ad7f27 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:59.880906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:59.887185 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:59.887881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.888163 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.888442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.889269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:04:59.889578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:59.891063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:04:59.920038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 119/230] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:04:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:04:59.930734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:04:59.936342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e676fb1-eb03-43e1-ac06-87e046ad7f27 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:04:59.964551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:04:59.978697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec78a34-524a-4a5e-b5fc-f50f5ef27e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:04:59.980196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec78a34-524a-4a5e-b5fc-f50f5ef27e41 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:04:59.980509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec78a34-524a-4a5e-b5fc-f50f5ef27e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:04:59.980766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec78a34-524a-4a5e-b5fc-f50f5ef27e41 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf/users/fb79248689d5403d8af3fce64dbdebbc/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:04:59.983266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec78a34-524a-4a5e-b5fc-f50f5ef27e41 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8b586142aa493484ff06e0a70778cf, user_id=fb79248689d5403d8af3fce64dbdebbc, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:04:59.984741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec78a34-524a-4a5e-b5fc-f50f5ef27e41 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:00.004790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 112/231] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:04:59 2022] PUT /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf/users/fb79248689d5403d8af3fce64dbdebbc/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:05:00.078713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b65146-a958-46c2-a45f-ec7eb371fc55 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:00.117947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc976b-f89e-42c2-a8e7-8e5deba9e9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:00.119170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc976b-f89e-42c2-a8e7-8e5deba9e9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:00.119448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc976b-f89e-42c2-a8e7-8e5deba9e9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:00.119921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc976b-f89e-42c2-a8e7-8e5deba9e9d8 None admin] PATH_INFO: `/v3/users/fb79248689d5403d8af3fce64dbdebbc` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:00.121329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc976b-f89e-42c2-a8e7-8e5deba9e9d8 None admin] RBAC: Authorizing `identity:get_user(user_id=fb79248689d5403d8af3fce64dbdebbc)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:00.122853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc976b-f89e-42c2-a8e7-8e5deba9e9d8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:00.124593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 120/232] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:04:59 2022] GET /identity/v3/users/fb79248689d5403d8af3fce64dbdebbc => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:00.130335 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec78a34-524a-4a5e-b5fc-f50f5ef27e41 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:00.264819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:00.281793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec78a34-524a-4a5e-b5fc-f50f5ef27e41 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:00.319465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c82b1-0085-4e80-a362-c6621e622139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:00.320401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c82b1-0085-4e80-a362-c6621e622139 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:00.320692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c82b1-0085-4e80-a362-c6621e622139 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:00.320975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c82b1-0085-4e80-a362-c6621e622139 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:00.322219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c82b1-0085-4e80-a362-c6621e622139 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:00.323728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086c82b1-0085-4e80-a362-c6621e622139 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:00.325805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 113/233] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:00 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:00.334959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc976b-f89e-42c2-a8e7-8e5deba9e9d8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:00.461170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:00.466620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc976b-f89e-42c2-a8e7-8e5deba9e9d8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:00.510815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca44566-2ca2-4eab-ae04-632ea6c5b293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:00.511779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44566-2ca2-4eab-ae04-632ea6c5b293 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:00.512066 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44566-2ca2-4eab-ae04-632ea6c5b293 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:00.512389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca44566-2ca2-4eab-ae04-632ea6c5b293 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf/users/fb79248689d5403d8af3fce64dbdebbc/roles/89d5f9aaa6284371b4d5205fea92cba1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:00.516085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca44566-2ca2-4eab-ae04-632ea6c5b293 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8b586142aa493484ff06e0a70778cf, user_id=fb79248689d5403d8af3fce64dbdebbc, role_id=89d5f9aaa6284371b4d5205fea92cba1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:00.517581 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca44566-2ca2-4eab-ae04-632ea6c5b293 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:00.543097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 121/234] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:05:00 2022] PUT /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf/users/fb79248689d5403d8af3fce64dbdebbc/roles/89d5f9aaa6284371b4d5205fea92cba1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:00.962006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c400af-6366-40f2-8761-f28a7dab46f5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:00.962500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c400af-6366-40f2-8761-f28a7dab46f5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:00.962500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c400af-6366-40f2-8761-f28a7dab46f5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:00.963938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 114/235] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:00.970562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f3cff-c370-4833-8ad5-5660e4d29617 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:00.971002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f3cff-c370-4833-8ad5-5660e4d29617 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:00.971181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f3cff-c370-4833-8ad5-5660e4d29617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:00.994091 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-742f3cff-c370-4833-8ad5-5660e4d29617 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:00.995430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-742f3cff-c370-4833-8ad5-5660e4d29617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:01.086054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 122/236] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:01.090859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85071119-1426-4452-b7ea-272291b7c5b1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.091152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85071119-1426-4452-b7ea-272291b7c5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.091434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85071119-1426-4452-b7ea-272291b7c5b1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.092828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 115/237] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:01.098188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6130e-7c9a-4a24-8bd2-1e49f80ff00f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.098478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6130e-7c9a-4a24-8bd2-1e49f80ff00f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.098757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6130e-7c9a-4a24-8bd2-1e49f80ff00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.122764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d1a6130e-7c9a-4a24-8bd2-1e49f80ff00f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:01.123983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d1a6130e-7c9a-4a24-8bd2-1e49f80ff00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:01.158878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85071119-1426-4452-b7ea-272291b7c5b1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:01.230471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 123/238] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:01.267024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6130e-7c9a-4a24-8bd2-1e49f80ff00f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:01.296469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:01.353244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85071119-1426-4452-b7ea-272291b7c5b1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:01.416879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d55db-f14f-4325-a1fc-15c48c96c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:01.419224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d55db-f14f-4325-a1fc-15c48c96c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.419903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d55db-f14f-4325-a1fc-15c48c96c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.420983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d55db-f14f-4325-a1fc-15c48c96c1e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.444017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d55db-f14f-4325-a1fc-15c48c96c1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-103d55db-f14f-4325-a1fc-15c48c96c1e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:01.455905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:01.457003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:01.458036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:01.463259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 116/239] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:01.469528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d55db-f14f-4325-a1fc-15c48c96c1e4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:01.481780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6130e-7c9a-4a24-8bd2-1e49f80ff00f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:01.509037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:01.523836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca6423-6f42-4aa0-8c16-6745cf176233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:01.524717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6423-6f42-4aa0-8c16-6745cf176233 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.525061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6423-6f42-4aa0-8c16-6745cf176233 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.525396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca6423-6f42-4aa0-8c16-6745cf176233 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.526951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca6423-6f42-4aa0-8c16-6745cf176233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:01.528814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ca6423-6f42-4aa0-8c16-6745cf176233 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:01.530992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 124/240] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:01.538739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca6423-6f42-4aa0-8c16-6745cf176233 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:01.614081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d55db-f14f-4325-a1fc-15c48c96c1e4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:01.657273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:01.658418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.659757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.662605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.663601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:01.664115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:01.665714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:01.682040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 117/241] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:01.690753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:01.719424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:01.736586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca6423-6f42-4aa0-8c16-6745cf176233 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:01.774158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18076218-f444-440e-bc73-f6767cc14951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:01.774994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076218-f444-440e-bc73-f6767cc14951 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.775370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076218-f444-440e-bc73-f6767cc14951 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.775875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076218-f444-440e-bc73-f6767cc14951 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.776920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18076218-f444-440e-bc73-f6767cc14951 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:01.778488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18076218-f444-440e-bc73-f6767cc14951 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:01.799742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:01.802477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 125/242] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:05:01 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:05:01.804043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83026ab6-2c0a-4141-88f0-e83b9bcd25c9 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.804460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83026ab6-2c0a-4141-88f0-e83b9bcd25c9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.804803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83026ab6-2c0a-4141-88f0-e83b9bcd25c9 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.806575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 126/243] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:01.824673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4d9f3-7182-498b-907c-54ddf0690d63 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.824993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4d9f3-7182-498b-907c-54ddf0690d63 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.825236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4d9f3-7182-498b-907c-54ddf0690d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.840573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c88aa89-c4c9-4c3d-9218-aecfe2253cea None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:01.844932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-24c4d9f3-7182-498b-907c-54ddf0690d63 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:01.846387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-24c4d9f3-7182-498b-907c-54ddf0690d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:01.914129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e94ca-60c5-4749-b76c-34e9461f8609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:01.915058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e94ca-60c5-4749-b76c-34e9461f8609 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:01.915343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e94ca-60c5-4749-b76c-34e9461f8609 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:01.915620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e94ca-60c5-4749-b76c-34e9461f8609 None admin] PATH_INFO: `/v3/users/b75901fe629844feb85c4cf9d24a043e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:01.931791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 127/244] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 18 14:05:01.940604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e94ca-60c5-4749-b76c-34e9461f8609 None admin] RBAC: Authorizing `identity:get_user(user_id=b75901fe629844feb85c4cf9d24a043e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:01.942158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e94ca-60c5-4749-b76c-34e9461f8609 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:01.943980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 118/245] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:01 2022] GET /identity/v3/users/b75901fe629844feb85c4cf9d24a043e => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:01.950113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4d9f3-7182-498b-907c-54ddf0690d63 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:01.978677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e94ca-60c5-4749-b76c-34e9461f8609 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:02.071350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:02.119679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4d9f3-7182-498b-907c-54ddf0690d63 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:02.154328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e94ca-60c5-4749-b76c-34e9461f8609 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:02.167480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d7f2e-1121-4cfb-8fae-b64acd1aa473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:02.168515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7f2e-1121-4cfb-8fae-b64acd1aa473 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.168805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7f2e-1121-4cfb-8fae-b64acd1aa473 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.169416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7f2e-1121-4cfb-8fae-b64acd1aa473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.201299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358218ca-4269-4a12-af9a-415bea51a603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:02.202165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358218ca-4269-4a12-af9a-415bea51a603 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.202448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358218ca-4269-4a12-af9a-415bea51a603 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.202727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358218ca-4269-4a12-af9a-415bea51a603 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.210961 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:02.216981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358218ca-4269-4a12-af9a-415bea51a603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-358218ca-4269-4a12-af9a-415bea51a603 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:02.225872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:02.226801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:02.227584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 119/246] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:02.235594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358218ca-4269-4a12-af9a-415bea51a603 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:02.267798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d7f2e-1121-4cfb-8fae-b64acd1aa473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:02.269793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d7f2e-1121-4cfb-8fae-b64acd1aa473 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:02.376418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:02.392541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358218ca-4269-4a12-af9a-415bea51a603 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:02.423916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 128/247] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:02.429310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d7f2e-1121-4cfb-8fae-b64acd1aa473 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:02.431087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:02.432203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.432594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.432879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.433672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:02.434008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:02.437677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:02.448595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 120/248] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:02.455900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:02.525097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:02.563152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d7f2e-1121-4cfb-8fae-b64acd1aa473 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:02.572497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328eb8e-a263-4bf9-a275-ee8f2379ceb5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:02.601435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab3ae9-c939-4bb1-9a02-5bdabd966d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:02.602684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab3ae9-c939-4bb1-9a02-5bdabd966d5d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.603226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab3ae9-c939-4bb1-9a02-5bdabd966d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.603722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab3ae9-c939-4bb1-9a02-5bdabd966d5d None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.606538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab3ae9-c939-4bb1-9a02-5bdabd966d5d None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:02.608951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ab3ae9-c939-4bb1-9a02-5bdabd966d5d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:02.610531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689043d1-8a4a-49a3-a438-42f8750eef7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:02.611453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689043d1-8a4a-49a3-a438-42f8750eef7b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.611768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 129/249] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:02 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:02.612134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689043d1-8a4a-49a3-a438-42f8750eef7b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.612421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689043d1-8a4a-49a3-a438-42f8750eef7b None admin] PATH_INFO: `/v3/users/a4d1597246dc498b9adf86d0a8670054` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.619019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689043d1-8a4a-49a3-a438-42f8750eef7b None admin] RBAC: Authorizing `identity:get_user(user_id=a4d1597246dc498b9adf86d0a8670054)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:02.619019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689043d1-8a4a-49a3-a438-42f8750eef7b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:02.619019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 121/250] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:02 2022] GET /identity/v3/users/a4d1597246dc498b9adf86d0a8670054 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:02.620777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab3ae9-c939-4bb1-9a02-5bdabd966d5d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:02.622752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689043d1-8a4a-49a3-a438-42f8750eef7b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:02.682919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:02.744755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689043d1-8a4a-49a3-a438-42f8750eef7b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:02.774332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ab3ae9-c939-4bb1-9a02-5bdabd966d5d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:02.800985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:02.814641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501403cb-9d19-4460-942c-f27222fdc23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:02.815536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501403cb-9d19-4460-942c-f27222fdc23b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.815830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501403cb-9d19-4460-942c-f27222fdc23b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.816111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501403cb-9d19-4460-942c-f27222fdc23b None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.817988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501403cb-9d19-4460-942c-f27222fdc23b None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:02.819684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501403cb-9d19-4460-942c-f27222fdc23b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:02.821607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 122/251] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:02 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:02.822765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eb379-533f-4364-a6ee-f9a30270aafe None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.823446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eb379-533f-4364-a6ee-f9a30270aafe None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.823446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eb379-533f-4364-a6ee-f9a30270aafe None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.831133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 123/252] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:02.836160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a72a7f-c274-44eb-a260-92f44c33be11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:02.836835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72a7f-c274-44eb-a260-92f44c33be11 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.837120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72a7f-c274-44eb-a260-92f44c33be11 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.837399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a72a7f-c274-44eb-a260-92f44c33be11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.838303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a72a7f-c274-44eb-a260-92f44c33be11 None admin] RBAC: Adding query filter params (role.id=6e98f7101fb9430a87421182e7b6f4a8, scope.project.id=5c8b586142aa493484ff06e0a70778cf, user.id=b75901fe629844feb85c4cf9d24a043e) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:02.838691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a72a7f-c274-44eb-a260-92f44c33be11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:02.840092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a72a7f-c274-44eb-a260-92f44c33be11 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:02.842246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eb379-533f-4364-a6ee-f9a30270aafe None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:02.857670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 130/253] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:02 2022] GET /identity/v3/role_assignments?user.id=b75901fe629844feb85c4cf9d24a043e&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=6e98f7101fb9430a87421182e7b6f4a8 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:02.858905 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424bb06-698e-4233-b554-ba7c8c3c5cfb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:02.859156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424bb06-698e-4233-b554-ba7c8c3c5cfb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:02.859435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424bb06-698e-4233-b554-ba7c8c3c5cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:02.896778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2424bb06-698e-4233-b554-ba7c8c3c5cfb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:02.897977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2424bb06-698e-4233-b554-ba7c8c3c5cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:03.023015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eb379-533f-4364-a6ee-f9a30270aafe None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:03.066546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d417a-9799-426f-89ea-4d2d4edcc54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:03.067826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d417a-9799-426f-89ea-4d2d4edcc54f None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:03.068355 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d417a-9799-426f-89ea-4d2d4edcc54f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:03.068879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d417a-9799-426f-89ea-4d2d4edcc54f None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf/users/a4d1597246dc498b9adf86d0a8670054/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:03.071164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d417a-9799-426f-89ea-4d2d4edcc54f None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8b586142aa493484ff06e0a70778cf, user_id=a4d1597246dc498b9adf86d0a8670054, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:03.073128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d417a-9799-426f-89ea-4d2d4edcc54f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:03.091509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 131/254] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:03.095891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 124/255] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:05:02 2022] PUT /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf/users/a4d1597246dc498b9adf86d0a8670054/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:03.127441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424bb06-698e-4233-b554-ba7c8c3c5cfb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:03.194656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:03.292911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424bb06-698e-4233-b554-ba7c8c3c5cfb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:03.348645 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc27b67-e76c-4cd7-8e6b-5c8d3ec57da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:03.349473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27b67-e76c-4cd7-8e6b-5c8d3ec57da6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:03.349785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27b67-e76c-4cd7-8e6b-5c8d3ec57da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:03.350066 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc27b67-e76c-4cd7-8e6b-5c8d3ec57da6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:03.364549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc27b67-e76c-4cd7-8e6b-5c8d3ec57da6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-4fc27b67-e76c-4cd7-8e6b-5c8d3ec57da6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:03.373038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:03.373969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:03.374766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 132/256] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:03.381099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d417a-9799-426f-89ea-4d2d4edcc54f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:03.501709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:03.528071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d66eb-30e5-40e0-bec4-3bcc3731b260 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:03.528496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d66eb-30e5-40e0-bec4-3bcc3731b260 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:03.528780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d66eb-30e5-40e0-bec4-3bcc3731b260 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:03.530520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 133/257] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:03.541543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4116f5a-1259-404b-b298-20419a5d77d5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:03.541956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4116f5a-1259-404b-b298-20419a5d77d5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:03.542241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4116f5a-1259-404b-b298-20419a5d77d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:03.569687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d417a-9799-426f-89ea-4d2d4edcc54f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:03.576490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e4116f5a-1259-404b-b298-20419a5d77d5 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:03.578940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e4116f5a-1259-404b-b298-20419a5d77d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:03.653072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:03.653786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:03.654078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:03.654357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:03.655118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:03.655429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:03.656831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:03.676187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 125/258] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:03.681735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:03.716796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 134/259] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:03.758357 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4116f5a-1259-404b-b298-20419a5d77d5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:03.798066 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:03.936686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1166a5-647c-4f06-9c95-1b110705c3c6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:03.958772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:03.968171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4116f5a-1259-404b-b298-20419a5d77d5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:03.983741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060e42e-4861-427b-8f65-54a98f3fe576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:03.985007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060e42e-4861-427b-8f65-54a98f3fe576 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:03.985510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060e42e-4861-427b-8f65-54a98f3fe576 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:03.986005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060e42e-4861-427b-8f65-54a98f3fe576 None admin] PATH_INFO: `/v3/users/fb79248689d5403d8af3fce64dbdebbc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:03.987479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060e42e-4861-427b-8f65-54a98f3fe576 None admin] RBAC: Authorizing `identity:get_user(user_id=fb79248689d5403d8af3fce64dbdebbc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:03.990991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060e42e-4861-427b-8f65-54a98f3fe576 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:03.995358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 126/260] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:03 2022] GET /identity/v3/users/fb79248689d5403d8af3fce64dbdebbc => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:03.998881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060e42e-4861-427b-8f65-54a98f3fe576 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:04.044461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22a07b-dd11-4545-b8e3-163a0e34da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:04.045649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22a07b-dd11-4545-b8e3-163a0e34da3e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:04.046148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22a07b-dd11-4545-b8e3-163a0e34da3e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:04.046614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22a07b-dd11-4545-b8e3-163a0e34da3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:04.067245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b22a07b-dd11-4545-b8e3-163a0e34da3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-3b22a07b-dd11-4545-b8e3-163a0e34da3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:04.080918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:04.081888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:04.082679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 135/261] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:05:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:04.087453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22a07b-dd11-4545-b8e3-163a0e34da3e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:04.184164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060e42e-4861-427b-8f65-54a98f3fe576 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:04.246185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedea24-9d08-4ef0-87d6-047f0eed91bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:04.246853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedea24-9d08-4ef0-87d6-047f0eed91bd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:04.247141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedea24-9d08-4ef0-87d6-047f0eed91bd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:04.247414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eedea24-9d08-4ef0-87d6-047f0eed91bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:04.248479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:04.303675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22a07b-dd11-4545-b8e3-163a0e34da3e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:04.310221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:04.330121 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedea24-9d08-4ef0-87d6-047f0eed91bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:04.331895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eedea24-9d08-4ef0-87d6-047f0eed91bd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:04.413160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:04.414323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:04.414938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:04.415613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:04.416824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:04.417376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:04.419056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:04.444292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 136/262] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:04.468353 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:04.515925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 127/263] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:04.521807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedea24-9d08-4ef0-87d6-047f0eed91bd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:04.596966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faec1c1-69dd-4830-81f6-85bf5875a516 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:04.664173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3c53c-113d-4f2a-88c7-7ac15fd77d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:04.665462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c53c-113d-4f2a-88c7-7ac15fd77d7c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:04.666096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c53c-113d-4f2a-88c7-7ac15fd77d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:04.666787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3c53c-113d-4f2a-88c7-7ac15fd77d7c None admin] PATH_INFO: `/v3/users/fb79248689d5403d8af3fce64dbdebbc` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:04.668586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3c53c-113d-4f2a-88c7-7ac15fd77d7c None admin] RBAC: Authorizing `identity:get_user(user_id=fb79248689d5403d8af3fce64dbdebbc)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:04.670502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3c53c-113d-4f2a-88c7-7ac15fd77d7c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:04.672766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 137/264] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:04 2022] GET /identity/v3/users/fb79248689d5403d8af3fce64dbdebbc => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:04.678582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3c53c-113d-4f2a-88c7-7ac15fd77d7c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:04.702838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:04.710693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:04.741477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eedea24-9d08-4ef0-87d6-047f0eed91bd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:04.834445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262ce2c-3eb2-4c59-8705-a323068f98e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:04.835354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262ce2c-3eb2-4c59-8705-a323068f98e6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:04.835642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262ce2c-3eb2-4c59-8705-a323068f98e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:04.835923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262ce2c-3eb2-4c59-8705-a323068f98e6 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:04.837228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262ce2c-3eb2-4c59-8705-a323068f98e6 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:04.838926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262ce2c-3eb2-4c59-8705-a323068f98e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:04.841041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 128/265] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:04 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:04.846819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262ce2c-3eb2-4c59-8705-a323068f98e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:04.880774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3c53c-113d-4f2a-88c7-7ac15fd77d7c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:04.931383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1affa0-f4e5-459e-9b97-6f43a562b825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:04.932491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1affa0-f4e5-459e-9b97-6f43a562b825 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:04.932972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1affa0-f4e5-459e-9b97-6f43a562b825 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:04.933465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1affa0-f4e5-459e-9b97-6f43a562b825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.041894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1affa0-f4e5-459e-9b97-6f43a562b825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:05.043891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1affa0-f4e5-459e-9b97-6f43a562b825 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:05.077778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:05.139975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262ce2c-3eb2-4c59-8705-a323068f98e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:05.169914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 138/266] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:05.175608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1affa0-f4e5-459e-9b97-6f43a562b825 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:05.258415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:05.259141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.259589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.260006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.261255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=5c8b586142aa493484ff06e0a70778cf, user.id=fb79248689d5403d8af3fce64dbdebbc) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:05.261740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:05.263374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:05.283816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 129/267] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:04 2022] GET /identity/v3/role_assignments?user.id=fb79248689d5403d8af3fce64dbdebbc&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=5c672a1b2979444383c587a215adeb35 => generated 608 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:05.284974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1affa0-f4e5-459e-9b97-6f43a562b825 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:05.305613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:05.335002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225b0bb-b3a2-4223-927e-f1a25a90ffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:05.335902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225b0bb-b3a2-4223-927e-f1a25a90ffb8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.336186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225b0bb-b3a2-4223-927e-f1a25a90ffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.336464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225b0bb-b3a2-4223-927e-f1a25a90ffb8 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.337815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225b0bb-b3a2-4223-927e-f1a25a90ffb8 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:05.339827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225b0bb-b3a2-4223-927e-f1a25a90ffb8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:05.342258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 139/268] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:05 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:05.354897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:05.399296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:05.408889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae7702-4324-4032-8347-e2f8053ed305 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.409516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae7702-4324-4032-8347-e2f8053ed305 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.410022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae7702-4324-4032-8347-e2f8053ed305 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.411893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 140/269] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:05.425015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088c672-b9d4-47c3-851a-69533b6496f7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.425649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088c672-b9d4-47c3-851a-69533b6496f7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.426332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088c672-b9d4-47c3-851a-69533b6496f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.472672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7088c672-b9d4-47c3-851a-69533b6496f7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:05.474151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7088c672-b9d4-47c3-851a-69533b6496f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:05.578735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb96721-7f37-40e9-9e94-0a471e4686b5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:05.579424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 141/270] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:05.581397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae2a0-7390-43d0-9bf8-bbba14c3ce4d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.582920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae2a0-7390-43d0-9bf8-bbba14c3ce4d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.583355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ae2a0-7390-43d0-9bf8-bbba14c3ce4d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.584880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 142/271] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:05.592101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34960-59e6-4f03-ba8a-07bae5adf84c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.592639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34960-59e6-4f03-ba8a-07bae5adf84c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.593062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f34960-59e6-4f03-ba8a-07bae5adf84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.625077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-90f34960-59e6-4f03-ba8a-07bae5adf84c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:05.626870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-90f34960-59e6-4f03-ba8a-07bae5adf84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:05.644522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:05.645406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.645870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.646313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.647309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] RBAC: Adding query filter params (role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=5c8b586142aa493484ff06e0a70778cf, user.id=fb79248689d5403d8af3fce64dbdebbc) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:05.647785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:05.649483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:05.660266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 130/272] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:05 2022] GET /identity/v3/role_assignments?user.id=fb79248689d5403d8af3fce64dbdebbc&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:05.666840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:05.721984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 143/273] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:05.748999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f34960-59e6-4f03-ba8a-07bae5adf84c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:05.807189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:05.821554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:05.847589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06c209-3229-4ba1-87c2-4654f31adc07 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:05.890219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f34960-59e6-4f03-ba8a-07bae5adf84c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:05.900507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868393f-e56e-43d5-940b-0aa99705222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:05.901844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868393f-e56e-43d5-940b-0aa99705222f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.902366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868393f-e56e-43d5-940b-0aa99705222f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.902837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868393f-e56e-43d5-940b-0aa99705222f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.904348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868393f-e56e-43d5-940b-0aa99705222f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:05.906160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868393f-e56e-43d5-940b-0aa99705222f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:05.908446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 131/274] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:05.910040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62e917-e19e-4671-aa66-4d334b06b9ba None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.910532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62e917-e19e-4671-aa66-4d334b06b9ba None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.911000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62e917-e19e-4671-aa66-4d334b06b9ba None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.912588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 132/275] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:05.918476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926dd89-dd42-450d-9d6c-700653c85631 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.919095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926dd89-dd42-450d-9d6c-700653c85631 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.919617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926dd89-dd42-450d-9d6c-700653c85631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.942484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6926dd89-dd42-450d-9d6c-700653c85631 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:05.944108 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436329-7139-4e0c-ad6d-5c31f46ca6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:05.944441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6926dd89-dd42-450d-9d6c-700653c85631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:05.944931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436329-7139-4e0c-ad6d-5c31f46ca6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:05.945216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436329-7139-4e0c-ad6d-5c31f46ca6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:05.945488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436329-7139-4e0c-ad6d-5c31f46ca6d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:05.957820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84436329-7139-4e0c-ad6d-5c31f46ca6d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-84436329-7139-4e0c-ad6d-5c31f46ca6d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:05.967044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:05.967929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:05.968706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:05.971705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 144/276] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:05.972521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436329-7139-4e0c-ad6d-5c31f46ca6d8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.037011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 133/277] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:06.038199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926dd89-dd42-450d-9d6c-700653c85631 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.103300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436329-7139-4e0c-ad6d-5c31f46ca6d8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:06.105115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:06.142113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed68f3-99e5-41a9-b5b6-43d7efa82cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:06.143249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed68f3-99e5-41a9-b5b6-43d7efa82cff None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.143748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed68f3-99e5-41a9-b5b6-43d7efa82cff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.144247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed68f3-99e5-41a9-b5b6-43d7efa82cff None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.145441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:06.145641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed68f3-99e5-41a9-b5b6-43d7efa82cff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:06.148100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ed68f3-99e5-41a9-b5b6-43d7efa82cff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:06.170342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6926dd89-dd42-450d-9d6c-700653c85631 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:06.171230 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 145/278] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:05:05 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:05:06.172216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed68f3-99e5-41a9-b5b6-43d7efa82cff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.223302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab8ea3-f9a0-4653-b709-cc8f97981f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:06.224008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab8ea3-f9a0-4653-b709-cc8f97981f93 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.224292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab8ea3-f9a0-4653-b709-cc8f97981f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.224857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab8ea3-f9a0-4653-b709-cc8f97981f93 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.225644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab8ea3-f9a0-4653-b709-cc8f97981f93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:06.226006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab8ea3-f9a0-4653-b709-cc8f97981f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:06.227511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab8ea3-f9a0-4653-b709-cc8f97981f93 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:06.237106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 134/279] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:06.238794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed7ddc-6acf-4261-a57b-c1889755b8ac None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.239227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed7ddc-6acf-4261-a57b-c1889755b8ac None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.239650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed7ddc-6acf-4261-a57b-c1889755b8ac None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.241073 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 135/280] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:06.243833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed7ddc-6acf-4261-a57b-c1889755b8ac None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.300693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed68f3-99e5-41a9-b5b6-43d7efa82cff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:06.345366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b3696-5c92-4df3-b977-28c3eb915d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:06.346211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b3696-5c92-4df3-b977-28c3eb915d3c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.346478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b3696-5c92-4df3-b977-28c3eb915d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.346750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b3696-5c92-4df3-b977-28c3eb915d3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.364358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b3696-5c92-4df3-b977-28c3eb915d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-212b3696-5c92-4df3-b977-28c3eb915d3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:06.372692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:06.373802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:06.374604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 146/281] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:06.375763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5646f89-fb8e-4525-a482-5f2fa17c59c1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.375763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5646f89-fb8e-4525-a482-5f2fa17c59c1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.375935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5646f89-fb8e-4525-a482-5f2fa17c59c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.378166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed7ddc-6acf-4261-a57b-c1889755b8ac None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:06.394663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a5646f89-fb8e-4525-a482-5f2fa17c59c1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:06.395864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a5646f89-fb8e-4525-a482-5f2fa17c59c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:06.411406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:06.429627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2198d4a-c574-4c4a-8f3e-6a1b7f09c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:06.430372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2198d4a-c574-4c4a-8f3e-6a1b7f09c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.430655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2198d4a-c574-4c4a-8f3e-6a1b7f09c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.430938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2198d4a-c574-4c4a-8f3e-6a1b7f09c5f6 None admin] PATH_INFO: `/v3/users/9ef71da16b384bb5984f2c932746f48c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.446791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:06.451174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2198d4a-c574-4c4a-8f3e-6a1b7f09c5f6 None admin] RBAC: Authorizing `identity:get_user(user_id=9ef71da16b384bb5984f2c932746f48c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:06.452674 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2198d4a-c574-4c4a-8f3e-6a1b7f09c5f6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:06.454653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 136/282] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/users/9ef71da16b384bb5984f2c932746f48c => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:06.455430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2198d4a-c574-4c4a-8f3e-6a1b7f09c5f6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.484852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 147/283] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:06.485878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5646f89-fb8e-4525-a482-5f2fa17c59c1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.589396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2198d4a-c574-4c4a-8f3e-6a1b7f09c5f6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:06.610415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5646f89-fb8e-4525-a482-5f2fa17c59c1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:06.648417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:06.649100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.649381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.649675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.650707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:06.651078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:06.653192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:06.659973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:06.660965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.661461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.661978 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.663083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] RBAC: Adding query filter params (role.id=9d6a699c2b1b45629628fd317cc9c15b, user.id=9ef71da16b384bb5984f2c932746f48c, scope.system=all) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:06.663318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 137/284] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:06.663790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:06.664218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.665898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:06.673087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 148/285] 10.222.0.71 () {56 vars in 1308 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/role_assignments?user.id=9ef71da16b384bb5984f2c932746f48c&scope.system=all&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:06.674438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.691510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:06.751427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:06.803920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0fc2e-5e70-480a-9991-0fd874716923 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:06.827464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd29c8-fecc-4fe0-89da-cb0842b98cfd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:06.877421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf5ec3-0cfd-467e-85f5-1648e30b2fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:06.879121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf5ec3-0cfd-467e-85f5-1648e30b2fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.879621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf5ec3-0cfd-467e-85f5-1648e30b2fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.880021 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf5ec3-0cfd-467e-85f5-1648e30b2fe3 None admin] PATH_INFO: `/v3/users/b75901fe629844feb85c4cf9d24a043e` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.882029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf5ec3-0cfd-467e-85f5-1648e30b2fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=b75901fe629844feb85c4cf9d24a043e)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:06.883689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acc651-784a-45aa-9e18-7b7da3cad529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:06.884794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc651-784a-45aa-9e18-7b7da3cad529 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:06.884910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf5ec3-0cfd-467e-85f5-1648e30b2fe3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:06.885452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc651-784a-45aa-9e18-7b7da3cad529 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:06.885998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc651-784a-45aa-9e18-7b7da3cad529 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:06.888803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 149/286] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/users/b75901fe629844feb85c4cf9d24a043e => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:06.900198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf5ec3-0cfd-467e-85f5-1648e30b2fe3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:06.916155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acc651-784a-45aa-9e18-7b7da3cad529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-72acc651-784a-45aa-9e18-7b7da3cad529 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:06.928585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:06.930676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:06.932585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 138/287] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:06.943229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acc651-784a-45aa-9e18-7b7da3cad529 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:07.080173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:07.126683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf5ec3-0cfd-467e-85f5-1648e30b2fe3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:07.182394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acc651-784a-45aa-9e18-7b7da3cad529 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:07.218404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:07.219531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38497575-1087-402b-854d-cd7529a9efe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:07.220394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497575-1087-402b-854d-cd7529a9efe5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:07.220712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497575-1087-402b-854d-cd7529a9efe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:07.221063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38497575-1087-402b-854d-cd7529a9efe5 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:07.222528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497575-1087-402b-854d-cd7529a9efe5 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:07.224183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38497575-1087-402b-854d-cd7529a9efe5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:07.229872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 150/288] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:07.245749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38497575-1087-402b-854d-cd7529a9efe5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:07.258516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:07.259501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:07.259982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:07.260525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:07.261549 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:07.262032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:07.263640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:07.284176 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 139/289] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:07.289988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:07.380500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38497575-1087-402b-854d-cd7529a9efe5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:07.427498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cadf7d-bf75-435d-8e66-784c96a75f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:07.429201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cadf7d-bf75-435d-8e66-784c96a75f6e None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:07.429718 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cadf7d-bf75-435d-8e66-784c96a75f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:07.430743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cadf7d-bf75-435d-8e66-784c96a75f6e None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf/users/b75901fe629844feb85c4cf9d24a043e/roles/6e98f7101fb9430a87421182e7b6f4a8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:07.434101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cadf7d-bf75-435d-8e66-784c96a75f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8b586142aa493484ff06e0a70778cf, user_id=b75901fe629844feb85c4cf9d24a043e, role_id=6e98f7101fb9430a87421182e7b6f4a8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:07.436496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cadf7d-bf75-435d-8e66-784c96a75f6e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:07.458891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 151/290] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:05:07 2022] PUT /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf/users/b75901fe629844feb85c4cf9d24a043e/roles/6e98f7101fb9430a87421182e7b6f4a8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:05:07.529140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fdfe48-65e5-4f6b-ae42-e99f23279ddc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:07.583693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19278-550f-4c33-a9bf-a9690a5f6ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:07.584617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19278-550f-4c33-a9bf-a9690a5f6ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:07.584899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19278-550f-4c33-a9bf-a9690a5f6ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:07.585172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19278-550f-4c33-a9bf-a9690a5f6ad2 None admin] PATH_INFO: `/v3/users/a4d1597246dc498b9adf86d0a8670054` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:07.586522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19278-550f-4c33-a9bf-a9690a5f6ad2 None admin] RBAC: Authorizing `identity:get_user(user_id=a4d1597246dc498b9adf86d0a8670054)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:07.587988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19278-550f-4c33-a9bf-a9690a5f6ad2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:07.590166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 140/291] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:07 2022] GET /identity/v3/users/a4d1597246dc498b9adf86d0a8670054 => generated 297 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:07.596099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cadf7d-bf75-435d-8e66-784c96a75f6e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:07.620506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:07.765176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cadf7d-bf75-435d-8e66-784c96a75f6e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:07.810505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897d27c-a772-4e04-9f7e-0f0be3e737c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:07.811110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d27c-a772-4e04-9f7e-0f0be3e737c1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:07.811397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d27c-a772-4e04-9f7e-0f0be3e737c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:07.811693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897d27c-a772-4e04-9f7e-0f0be3e737c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:07.888911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897d27c-a772-4e04-9f7e-0f0be3e737c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:07.890443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897d27c-a772-4e04-9f7e-0f0be3e737c1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:07.900855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:07.999382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 152/292] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:08.005065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19278-550f-4c33-a9bf-a9690a5f6ad2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:08.095900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:08.170578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19278-550f-4c33-a9bf-a9690a5f6ad2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:08.223832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f3b0e-dafe-4c37-b4cf-7e4dc82d035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:08.225138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f3b0e-dafe-4c37-b4cf-7e4dc82d035d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:08.225694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f3b0e-dafe-4c37-b4cf-7e4dc82d035d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:08.226159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f3b0e-dafe-4c37-b4cf-7e4dc82d035d None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:08.227708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f3b0e-dafe-4c37-b4cf-7e4dc82d035d None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:08.229499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f3b0e-dafe-4c37-b4cf-7e4dc82d035d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:08.231765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 141/293] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:08 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:08.238195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897d27c-a772-4e04-9f7e-0f0be3e737c1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:08.386216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897d27c-a772-4e04-9f7e-0f0be3e737c1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:08.402308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:08.438965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae5e9f-8347-42d6-9334-295c831cd1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:08.440004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae5e9f-8347-42d6-9334-295c831cd1ca None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:08.440504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae5e9f-8347-42d6-9334-295c831cd1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:08.440999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae5e9f-8347-42d6-9334-295c831cd1ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:08.442047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae5e9f-8347-42d6-9334-295c831cd1ca None admin] RBAC: Adding query filter params (role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=5c8b586142aa493484ff06e0a70778cf, user.id=a4d1597246dc498b9adf86d0a8670054) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:08.442560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae5e9f-8347-42d6-9334-295c831cd1ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:08.444235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae5e9f-8347-42d6-9334-295c831cd1ca None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:08.454679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 153/294] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:08 2022] GET /identity/v3/role_assignments?user.id=a4d1597246dc498b9adf86d0a8670054&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:08.820074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429b168-ea6e-44cc-af8f-2642cff289cb None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:08.820929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429b168-ea6e-44cc-af8f-2642cff289cb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:08.821355 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429b168-ea6e-44cc-af8f-2642cff289cb None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:08.823361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 142/295] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:08.830420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b8c1-dfc1-4e33-9da1-e338025f7267 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:08.830830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b8c1-dfc1-4e33-9da1-e338025f7267 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:08.831173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902b8c1-dfc1-4e33-9da1-e338025f7267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:08.849724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1902b8c1-dfc1-4e33-9da1-e338025f7267 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:08.851335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1902b8c1-dfc1-4e33-9da1-e338025f7267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:08.936620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 154/296] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:08.960096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429b168-ea6e-44cc-af8f-2642cff289cb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.088337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429b168-ea6e-44cc-af8f-2642cff289cb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:09.105881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30a7fe-eece-415b-b953-182f1f719bbc None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.106173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30a7fe-eece-415b-b953-182f1f719bbc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.106482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30a7fe-eece-415b-b953-182f1f719bbc None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.108576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 155/297] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:09.113169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:09.114874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c08a1-c8e3-40a3-b574-b4d044113b38 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.115172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c08a1-c8e3-40a3-b574-b4d044113b38 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.115451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c08a1-c8e3-40a3-b574-b4d044113b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.129345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9f4c9-5b89-4c74-92cd-8479244cc874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:09.130590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f4c9-5b89-4c74-92cd-8479244cc874 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.131066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f4c9-5b89-4c74-92cd-8479244cc874 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.131533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f4c9-5b89-4c74-92cd-8479244cc874 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.133984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9f4c9-5b89-4c74-92cd-8479244cc874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:09.135979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9f4c9-5b89-4c74-92cd-8479244cc874 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:09.137971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 143/298] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:09.143835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3b3c08a1-c8e3-40a3-b574-b4d044113b38 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:09.145383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3b3c08a1-c8e3-40a3-b574-b4d044113b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:09.146487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9f4c9-5b89-4c74-92cd-8479244cc874 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.237242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 156/299] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:09.259232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9f4c9-5b89-4c74-92cd-8479244cc874 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:09.263768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c08a1-c8e3-40a3-b574-b4d044113b38 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.301937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ce22c-16e0-4b04-ba80-4de2fd95bcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:09.302724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ce22c-16e0-4b04-ba80-4de2fd95bcd7 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.303076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ce22c-16e0-4b04-ba80-4de2fd95bcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.303176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:09.303450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ce22c-16e0-4b04-ba80-4de2fd95bcd7 None admin] PATH_INFO: `/v3/groups` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.304308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ce22c-16e0-4b04-ba80-4de2fd95bcd7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:09.305877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ce22c-16e0-4b04-ba80-4de2fd95bcd7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:09.323270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 144/300] 10.222.0.71 () {60 vars in 1134 bytes} [Mon Jul 18 14:05:09 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:05:09.325031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ffccf-04ba-4581-8703-07e19575a3b8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.325589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ffccf-04ba-4581-8703-07e19575a3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.326099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ffccf-04ba-4581-8703-07e19575a3b8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.328038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 145/301] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:09.334085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8930a-2a80-49a7-9179-9f8855041aa2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.334758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8930a-2a80-49a7-9179-9f8855041aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.335238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b8930a-2a80-49a7-9179-9f8855041aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.355010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-64b8930a-2a80-49a7-9179-9f8855041aa2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:09.356933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-64b8930a-2a80-49a7-9179-9f8855041aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:09.408676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c08a1-c8e3-40a3-b574-b4d044113b38 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:09.422299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:09.446363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 146/302] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:09.465862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02f3c3-5a79-4064-82cb-e71e27e2fe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:09.467062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02f3c3-5a79-4064-82cb-e71e27e2fe5f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.467618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02f3c3-5a79-4064-82cb-e71e27e2fe5f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.468115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02f3c3-5a79-4064-82cb-e71e27e2fe5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.471914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8930a-2a80-49a7-9179-9f8855041aa2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.486474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02f3c3-5a79-4064-82cb-e71e27e2fe5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0c02f3c3-5a79-4064-82cb-e71e27e2fe5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:09.497231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:09.498263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:09.499043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 157/303] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:09.505054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02f3c3-5a79-4064-82cb-e71e27e2fe5f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.610680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b8930a-2a80-49a7-9179-9f8855041aa2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:09.620994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:09.633749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02f3c3-5a79-4064-82cb-e71e27e2fe5f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:09.658631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee6cc7-fb47-4187-8df3-8a0ed3f83f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:09.659450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee6cc7-fb47-4187-8df3-8a0ed3f83f39 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.660281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee6cc7-fb47-4187-8df3-8a0ed3f83f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.660565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee6cc7-fb47-4187-8df3-8a0ed3f83f39 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.675433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee6cc7-fb47-4187-8df3-8a0ed3f83f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:09.676367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:09.677046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.677328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.677602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.678404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:09.678710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:09.680101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b4ee6cc7-fb47-4187-8df3-8a0ed3f83f39 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:09.684799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:09.685715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:09.686486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:09.687997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 147/304] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:09.692697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee6cc7-fb47-4187-8df3-8a0ed3f83f39 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.694211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 158/305] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:09.700502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.748304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:09.817700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c626c-f713-4ec9-94ed-14710c07fc1a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:09.829310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee6cc7-fb47-4187-8df3-8a0ed3f83f39 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:09.865515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328d877-719f-4d9a-b8ff-f4ddc8bd4c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:09.866452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328d877-719f-4d9a-b8ff-f4ddc8bd4c7a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.866739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328d877-719f-4d9a-b8ff-f4ddc8bd4c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.867030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328d877-719f-4d9a-b8ff-f4ddc8bd4c7a None admin] PATH_INFO: `/v3/users/62cbe777d2244a22997a6d8372402472` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.870900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:09.871906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:09.872385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:09.872899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:09.874032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:09.874542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:09.876129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:09.884986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328d877-719f-4d9a-b8ff-f4ddc8bd4c7a None admin] RBAC: Authorizing `identity:get_user(user_id=62cbe777d2244a22997a6d8372402472)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:09.886686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328d877-719f-4d9a-b8ff-f4ddc8bd4c7a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:09.887153 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 148/306] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:09.892773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 159/307] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:09 2022] GET /identity/v3/users/62cbe777d2244a22997a6d8372402472 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:09.894438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.895234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328d877-719f-4d9a-b8ff-f4ddc8bd4c7a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:09.914910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:10.013835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:10.017165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a261a-323e-468e-a1e6-83444f6d8c08 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:10.034237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328d877-719f-4d9a-b8ff-f4ddc8bd4c7a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:10.053290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a48c4-26e1-42a5-8d32-8d336df15d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:10.054202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a48c4-26e1-42a5-8d32-8d336df15d4a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.054489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a48c4-26e1-42a5-8d32-8d336df15d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.054764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a48c4-26e1-42a5-8d32-8d336df15d4a None admin] PATH_INFO: `/v3/users/9ef71da16b384bb5984f2c932746f48c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:10.056043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a48c4-26e1-42a5-8d32-8d336df15d4a None admin] RBAC: Authorizing `identity:get_user(user_id=9ef71da16b384bb5984f2c932746f48c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:10.057901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a48c4-26e1-42a5-8d32-8d336df15d4a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:10.064285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 149/308] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:09 2022] GET /identity/v3/users/9ef71da16b384bb5984f2c932746f48c => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:10.064285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c09ee-23f4-4940-8d5d-a83e8633b250 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.064285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c09ee-23f4-4940-8d5d-a83e8633b250 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.064285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c09ee-23f4-4940-8d5d-a83e8633b250 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:10.064285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 150/309] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:10.066801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c09ee-23f4-4940-8d5d-a83e8633b250 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:10.077105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45363e-efb3-4769-857a-3e9134ab249d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:10.077799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45363e-efb3-4769-857a-3e9134ab249d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.078081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45363e-efb3-4769-857a-3e9134ab249d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.078354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45363e-efb3-4769-857a-3e9134ab249d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:10.079183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45363e-efb3-4769-857a-3e9134ab249d None admin] RBAC: Adding query filter params (role.id=6e98f7101fb9430a87421182e7b6f4a8, user.id=62cbe777d2244a22997a6d8372402472, scope.system=all) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:10.079495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45363e-efb3-4769-857a-3e9134ab249d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:10.080958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45363e-efb3-4769-857a-3e9134ab249d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:10.086466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 160/310] 10.222.0.71 () {56 vars in 1308 bytes} [Mon Jul 18 14:05:09 2022] GET /identity/v3/role_assignments?user.id=62cbe777d2244a22997a6d8372402472&scope.system=all&role.id=6e98f7101fb9430a87421182e7b6f4a8 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:10.094214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105290d4-771a-4acf-9692-d144af11f40e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.094214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105290d4-771a-4acf-9692-d144af11f40e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.094214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105290d4-771a-4acf-9692-d144af11f40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:10.111457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-105290d4-771a-4acf-9692-d144af11f40e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:10.112595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-105290d4-771a-4acf-9692-d144af11f40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:10.205178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 161/311] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:10.232897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105290d4-771a-4acf-9692-d144af11f40e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:10.248883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:10.250075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c09ee-23f4-4940-8d5d-a83e8633b250 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:10.298869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49ce73-5f9b-4aa2-b5a9-c49c71caa3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:10.299762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49ce73-5f9b-4aa2-b5a9-c49c71caa3b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.300113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49ce73-5f9b-4aa2-b5a9-c49c71caa3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.300458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49ce73-5f9b-4aa2-b5a9-c49c71caa3b0 None admin] PATH_INFO: `/v3/system/users/9ef71da16b384bb5984f2c932746f48c/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:10.302886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49ce73-5f9b-4aa2-b5a9-c49c71caa3b0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9ef71da16b384bb5984f2c932746f48c, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:10.304620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49ce73-5f9b-4aa2-b5a9-c49c71caa3b0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:10.316516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 151/312] 10.222.0.71 () {58 vars in 1334 bytes} [Mon Jul 18 14:05:10 2022] PUT /identity/v3/system/users/9ef71da16b384bb5984f2c932746f48c/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:10.395335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105290d4-771a-4acf-9692-d144af11f40e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:10.438172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6f74-aff1-4998-b1ac-eeae454496ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:10.439127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6f74-aff1-4998-b1ac-eeae454496ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.439433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6f74-aff1-4998-b1ac-eeae454496ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.439714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba6f74-aff1-4998-b1ac-eeae454496ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:10.453677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba6f74-aff1-4998-b1ac-eeae454496ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8dba6f74-aff1-4998-b1ac-eeae454496ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:10.461889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:10.462885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:10.463719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:10.466894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 162/313] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:10.469801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49ce73-5f9b-4aa2-b5a9-c49c71caa3b0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:10.517165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:10.591099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49ce73-5f9b-4aa2-b5a9-c49c71caa3b0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:10.641579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:10.642585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.643061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.643563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:10.644527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:10.645040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:10.646654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:10.657524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 152/314] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:10.663000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6f74-aff1-4998-b1ac-eeae454496ff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:10.746810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:10.784382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba6f74-aff1-4998-b1ac-eeae454496ff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:10.820259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4b1bb-d507-4bd3-8170-d8073b210484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:10.821159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4b1bb-d507-4bd3-8170-d8073b210484 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.821443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4b1bb-d507-4bd3-8170-d8073b210484 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.821756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4b1bb-d507-4bd3-8170-d8073b210484 None admin] PATH_INFO: `/v3/users/b75901fe629844feb85c4cf9d24a043e` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:10.823097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4b1bb-d507-4bd3-8170-d8073b210484 None admin] RBAC: Authorizing `identity:get_user(user_id=b75901fe629844feb85c4cf9d24a043e)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:10.824841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4b1bb-d507-4bd3-8170-d8073b210484 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:10.826796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 163/315] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:10 2022] GET /identity/v3/users/b75901fe629844feb85c4cf9d24a043e => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:10.832400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:10.935725 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:10.943371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9a44a-f8d7-4d18-90e2-428534ce242e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:10.980204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ddd08-4832-4b66-9bd7-0aceb00bafe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:10.981603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ddd08-4832-4b66-9bd7-0aceb00bafe5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:10.982093 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ddd08-4832-4b66-9bd7-0aceb00bafe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:10.982518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ddd08-4832-4b66-9bd7-0aceb00bafe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:11.058585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ddd08-4832-4b66-9bd7-0aceb00bafe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:11.060075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ddd08-4832-4b66-9bd7-0aceb00bafe5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:11.166049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:11.173919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 153/316] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:11.180042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4b1bb-d507-4bd3-8170-d8073b210484 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:11.293832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4b1bb-d507-4bd3-8170-d8073b210484 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:11.332739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d47ace-e638-40cb-b303-fb9d499ee58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:11.334123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d47ace-e638-40cb-b303-fb9d499ee58e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:11.334638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d47ace-e638-40cb-b303-fb9d499ee58e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:11.335117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d47ace-e638-40cb-b303-fb9d499ee58e None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:11.336674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d47ace-e638-40cb-b303-fb9d499ee58e None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:11.338507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d47ace-e638-40cb-b303-fb9d499ee58e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:11.345432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 164/317] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:11 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:11.350882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ddd08-4832-4b66-9bd7-0aceb00bafe5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:11.452955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ddd08-4832-4b66-9bd7-0aceb00bafe5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:11.488013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e8c05-5951-4810-8a52-e84cab0021ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:11.489063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8c05-5951-4810-8a52-e84cab0021ca None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:11.489560 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8c05-5951-4810-8a52-e84cab0021ca None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:11.490059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e8c05-5951-4810-8a52-e84cab0021ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:11.491104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e8c05-5951-4810-8a52-e84cab0021ca None admin] RBAC: Adding query filter params (role.id=6e98f7101fb9430a87421182e7b6f4a8, scope.project.id=5c8b586142aa493484ff06e0a70778cf, user.id=b75901fe629844feb85c4cf9d24a043e) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:11.491619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e8c05-5951-4810-8a52-e84cab0021ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:11.493610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e8c05-5951-4810-8a52-e84cab0021ca None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:11.505790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 154/318] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:11 2022] GET /identity/v3/role_assignments?user.id=b75901fe629844feb85c4cf9d24a043e&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=6e98f7101fb9430a87421182e7b6f4a8 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:11.569667 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834edde-7ae9-4d75-998f-3673d720c056 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:11.570180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834edde-7ae9-4d75-998f-3673d720c056 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:11.570255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834edde-7ae9-4d75-998f-3673d720c056 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:11.572178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 165/319] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:11.579332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77606a-b688-49e8-a2e7-580a9547a0b4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:11.579837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77606a-b688-49e8-a2e7-580a9547a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:11.580120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a77606a-b688-49e8-a2e7-580a9547a0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:11.603047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4a77606a-b688-49e8-a2e7-580a9547a0b4 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:11.604328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4a77606a-b688-49e8-a2e7-580a9547a0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:11.685863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:11.705170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 155/320] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:11.728512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834edde-7ae9-4d75-998f-3673d720c056 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:11.732405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:11.851549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834edde-7ae9-4d75-998f-3673d720c056 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:11.888286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d11a7f-b730-47d7-a2c5-6d244f7cf9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:11.889094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d11a7f-b730-47d7-a2c5-6d244f7cf9cd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:11.889375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d11a7f-b730-47d7-a2c5-6d244f7cf9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:11.889648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d11a7f-b730-47d7-a2c5-6d244f7cf9cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:11.890844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d11a7f-b730-47d7-a2c5-6d244f7cf9cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:11.892298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d11a7f-b730-47d7-a2c5-6d244f7cf9cd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:11.894179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 166/321] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:11.899638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77606a-b688-49e8-a2e7-580a9547a0b4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:12.003872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a77606a-b688-49e8-a2e7-580a9547a0b4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:12.038189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26656b-bfb0-4617-957e-cb4f65b60a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:12.038865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26656b-bfb0-4617-957e-cb4f65b60a6b None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:12.039146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26656b-bfb0-4617-957e-cb4f65b60a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:12.039420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26656b-bfb0-4617-957e-cb4f65b60a6b None admin] PATH_INFO: `/v3/groups` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:12.040127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26656b-bfb0-4617-957e-cb4f65b60a6b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:12.041773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26656b-bfb0-4617-957e-cb4f65b60a6b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:12.058314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 156/322] 10.222.0.71 () {60 vars in 1134 bytes} [Mon Jul 18 14:05:11 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:05:12.295006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce1faf-45fa-4b12-9b07-160747185f77 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:12.295412 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce1faf-45fa-4b12-9b07-160747185f77 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:12.295536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce1faf-45fa-4b12-9b07-160747185f77 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:12.297104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 167/323] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:12.302493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215344c-d810-415d-a292-e2f72f794933 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:12.303052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215344c-d810-415d-a292-e2f72f794933 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:12.303528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215344c-d810-415d-a292-e2f72f794933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:12.322346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5215344c-d810-415d-a292-e2f72f794933 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:12.324386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5215344c-d810-415d-a292-e2f72f794933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:12.381949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:12.412231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 157/324] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:12.434290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce1faf-45fa-4b12-9b07-160747185f77 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:12.516082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:12.525436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316637ed-14c1-4de6-9a80-f116924b4fef None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:12.525800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316637ed-14c1-4de6-9a80-f116924b4fef None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:12.526625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316637ed-14c1-4de6-9a80-f116924b4fef None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:12.528882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 158/325] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:12.534150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc764716-e507-41d5-8f86-5fd7b42fff5e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:12.534888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc764716-e507-41d5-8f86-5fd7b42fff5e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:12.535365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc764716-e507-41d5-8f86-5fd7b42fff5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:12.559126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fc764716-e507-41d5-8f86-5fd7b42fff5e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:12.560913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fc764716-e507-41d5-8f86-5fd7b42fff5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:12.562511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce1faf-45fa-4b12-9b07-160747185f77 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:12.610854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b1e10-af13-4882-84bc-fa3b135354e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:12.612656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b1e10-af13-4882-84bc-fa3b135354e4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:12.613085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b1e10-af13-4882-84bc-fa3b135354e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:12.613787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b1e10-af13-4882-84bc-fa3b135354e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:12.629911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b1e10-af13-4882-84bc-fa3b135354e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:12.636385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 159/326] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-6a8b1e10-af13-4882-84bc-fa3b135354e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:12.646599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:12.647494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:12.648273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 168/327] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:12.648941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc764716-e507-41d5-8f86-5fd7b42fff5e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:12.660655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b1e10-af13-4882-84bc-fa3b135354e4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:12.778885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc764716-e507-41d5-8f86-5fd7b42fff5e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:12.789420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:12.815143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b1e10-af13-4882-84bc-fa3b135354e4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:12.827760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:12.828587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:12.828965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:12.829335 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:12.830238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:12.830643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:12.832138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:12.842269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 160/328] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:12.854070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:12.854792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:12.860185 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2078d95-b1b1-4ba5-8922-759acde74863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:12.861210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2078d95-b1b1-4ba5-8922-759acde74863 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:12.861631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2078d95-b1b1-4ba5-8922-759acde74863 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:12.862109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2078d95-b1b1-4ba5-8922-759acde74863 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:12.879871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2078d95-b1b1-4ba5-8922-759acde74863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a2078d95-b1b1-4ba5-8922-759acde74863 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:12.899702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:12.901064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:12.902003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:12.905773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 169/329] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:12.912749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2078d95-b1b1-4ba5-8922-759acde74863 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:13.038700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7420751-9635-4715-8c09-8ca99a65c49a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:13.104604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4afc4-f0c4-4cac-b22f-c900a9413480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:13.105814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4afc4-f0c4-4cac-b22f-c900a9413480 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:13.106267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4afc4-f0c4-4cac-b22f-c900a9413480 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:13.106734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4afc4-f0c4-4cac-b22f-c900a9413480 None admin] PATH_INFO: `/v3/users/62cbe777d2244a22997a6d8372402472` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:13.110522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4afc4-f0c4-4cac-b22f-c900a9413480 None admin] RBAC: Authorizing `identity:get_user(user_id=62cbe777d2244a22997a6d8372402472)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:13.112231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4afc4-f0c4-4cac-b22f-c900a9413480 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:13.114180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 161/330] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:12 2022] GET /identity/v3/users/62cbe777d2244a22997a6d8372402472 => generated 293 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:13.120630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4afc4-f0c4-4cac-b22f-c900a9413480 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:13.160919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2078d95-b1b1-4ba5-8922-759acde74863 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:13.206147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:13.236424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:13.237730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:13.239537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:13.240208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:13.242417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:13.242950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:13.245116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:13.261632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 170/331] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:13.277236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:13.347582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4afc4-f0c4-4cac-b22f-c900a9413480 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:13.385602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:13.411871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530789a-462f-4804-94a9-2b2838d5b449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:13.413064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530789a-462f-4804-94a9-2b2838d5b449 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:13.413550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530789a-462f-4804-94a9-2b2838d5b449 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:13.414046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530789a-462f-4804-94a9-2b2838d5b449 None admin] PATH_INFO: `/v3/system/users/62cbe777d2244a22997a6d8372402472/roles/6e98f7101fb9430a87421182e7b6f4a8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:13.417168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530789a-462f-4804-94a9-2b2838d5b449 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=62cbe777d2244a22997a6d8372402472, role_id=6e98f7101fb9430a87421182e7b6f4a8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:13.418974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530789a-462f-4804-94a9-2b2838d5b449 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:13.434735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 162/332] 10.222.0.71 () {58 vars in 1334 bytes} [Mon Jul 18 14:05:13 2022] PUT /identity/v3/system/users/62cbe777d2244a22997a6d8372402472/roles/6e98f7101fb9430a87421182e7b6f4a8 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:13.469409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcad50-bb35-4c44-a430-1270d437e89e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:13.515769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13282e5-35bd-4455-9a43-5cfc13ed334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:13.517755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13282e5-35bd-4455-9a43-5cfc13ed334d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:13.518524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13282e5-35bd-4455-9a43-5cfc13ed334d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:13.519261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13282e5-35bd-4455-9a43-5cfc13ed334d None admin] PATH_INFO: `/v3/users/9ef71da16b384bb5984f2c932746f48c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:13.521541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13282e5-35bd-4455-9a43-5cfc13ed334d None admin] RBAC: Authorizing `identity:get_user(user_id=9ef71da16b384bb5984f2c932746f48c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:13.523740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13282e5-35bd-4455-9a43-5cfc13ed334d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:13.528919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 171/333] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:13 2022] GET /identity/v3/users/9ef71da16b384bb5984f2c932746f48c => generated 293 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:13.535682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530789a-462f-4804-94a9-2b2838d5b449 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:13.717072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530789a-462f-4804-94a9-2b2838d5b449 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:13.718979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:13.782923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1af9e5-873a-4c9b-bca1-23a67adc68c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:13.783885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1af9e5-873a-4c9b-bca1-23a67adc68c8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:13.784378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1af9e5-873a-4c9b-bca1-23a67adc68c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:13.784866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1af9e5-873a-4c9b-bca1-23a67adc68c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:13.786519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1af9e5-873a-4c9b-bca1-23a67adc68c8 None admin] RBAC: Adding query filter params (role.id=9d6a699c2b1b45629628fd317cc9c15b, user.id=9ef71da16b384bb5984f2c932746f48c, scope.system=all) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:13.787071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1af9e5-873a-4c9b-bca1-23a67adc68c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:13.789694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1af9e5-873a-4c9b-bca1-23a67adc68c8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:13.818241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 163/334] 10.222.0.71 () {56 vars in 1308 bytes} [Mon Jul 18 14:05:13 2022] GET /identity/v3/role_assignments?user.id=9ef71da16b384bb5984f2c932746f48c&scope.system=all&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 510 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:14.824517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af5bd9-d53c-4ab0-9511-8c41f13b245b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:14.825895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af5bd9-d53c-4ab0-9511-8c41f13b245b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:14.826509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af5bd9-d53c-4ab0-9511-8c41f13b245b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:14.828413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 172/335] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:14.834548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad749-8e09-4dad-82d0-a024a7b485f8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:14.835673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad749-8e09-4dad-82d0-a024a7b485f8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:14.836183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ad749-8e09-4dad-82d0-a024a7b485f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:14.860038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b27ad749-8e09-4dad-82d0-a024a7b485f8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:14.862625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b27ad749-8e09-4dad-82d0-a024a7b485f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:14.910893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed00a65-3575-4c01-9f41-2bde58aad549 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:14.911498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed00a65-3575-4c01-9f41-2bde58aad549 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:14.911963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed00a65-3575-4c01-9f41-2bde58aad549 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:14.913819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 173/336] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:14.919853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3b3a-424d-4e86-bc87-be3d9a3e0f73 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:14.920441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3b3a-424d-4e86-bc87-be3d9a3e0f73 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:14.921007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c3b3a-424d-4e86-bc87-be3d9a3e0f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:14.946403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-748c3b3a-424d-4e86-bc87-be3d9a3e0f73 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:14.948326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-748c3b3a-424d-4e86-bc87-be3d9a3e0f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:15.001205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 164/337] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:15.052806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ad749-8e09-4dad-82d0-a024a7b485f8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:15.058829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 174/338] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:15.100603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c3b3a-424d-4e86-bc87-be3d9a3e0f73 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:15.172641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:15.200154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ad749-8e09-4dad-82d0-a024a7b485f8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:15.212858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:15.280245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40689790-ac68-4e8f-8f23-91e4a6bb12c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:15.281078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40689790-ac68-4e8f-8f23-91e4a6bb12c0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:15.281367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40689790-ac68-4e8f-8f23-91e4a6bb12c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:15.281646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40689790-ac68-4e8f-8f23-91e4a6bb12c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:15.309028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40689790-ac68-4e8f-8f23-91e4a6bb12c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-40689790-ac68-4e8f-8f23-91e4a6bb12c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:15.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:15.318329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:15.319113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:15.320472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 165/339] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:15.333000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c3b3a-424d-4e86-bc87-be3d9a3e0f73 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:15.334550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40689790-ac68-4e8f-8f23-91e4a6bb12c0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:15.411002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd9920-c6c1-45d7-9da1-18d30411988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:15.411843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9920-c6c1-45d7-9da1-18d30411988f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:15.412126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9920-c6c1-45d7-9da1-18d30411988f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:15.412400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd9920-c6c1-45d7-9da1-18d30411988f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:15.434880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd9920-c6c1-45d7-9da1-18d30411988f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-46fd9920-c6c1-45d7-9da1-18d30411988f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:15.451305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:15.452294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:15.453104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 175/340] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:05:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:15.466383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd9920-c6c1-45d7-9da1-18d30411988f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:15.565580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40689790-ac68-4e8f-8f23-91e4a6bb12c0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:15.659141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:15.659853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:15.660141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:15.660421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:15.661204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:15.661517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:15.663020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:15.681086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 166/341] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:15.688654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:15.720560 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:15.763564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd9920-c6c1-45d7-9da1-18d30411988f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:15.785481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:15.847676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:15.848848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:15.849364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:15.855298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:15.856968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:15.857525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:15.860917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:15.877527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 176/342] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:15.889822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:15.919715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc12ab-d85e-4caa-9b69-f2ff10c93680 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:15.980368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136c312-1693-4253-ad09-6bc7b9382b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:15.981356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c312-1693-4253-ad09-6bc7b9382b9c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:15.982302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c312-1693-4253-ad09-6bc7b9382b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:15.982302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c312-1693-4253-ad09-6bc7b9382b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:16.056184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5400dc5-7d72-4582-8e4f-a16b72835723 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:16.097714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136c312-1693-4253-ad09-6bc7b9382b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:16.098038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136c312-1693-4253-ad09-6bc7b9382b9c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:16.119832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53354feb-0d45-4752-a261-b51521de0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:16.120878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53354feb-0d45-4752-a261-b51521de0156 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:16.121493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53354feb-0d45-4752-a261-b51521de0156 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:16.122013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53354feb-0d45-4752-a261-b51521de0156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:16.128796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:16.194336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:16.252419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 167/343] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:05:16.258471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136c312-1693-4253-ad09-6bc7b9382b9c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:16.281172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53354feb-0d45-4752-a261-b51521de0156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:16.282698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53354feb-0d45-4752-a261-b51521de0156 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:16.399433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136c312-1693-4253-ad09-6bc7b9382b9c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:16.420340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:16.446048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1443cb-0842-47b1-9819-c2601f28c702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:16.447010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1443cb-0842-47b1-9819-c2601f28c702 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:16.447295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1443cb-0842-47b1-9819-c2601f28c702 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:16.447734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1443cb-0842-47b1-9819-c2601f28c702 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:16.449117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1443cb-0842-47b1-9819-c2601f28c702 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:16.450712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1443cb-0842-47b1-9819-c2601f28c702 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:16.452941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 168/344] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:16 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:16.454556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22bda-ab5d-48c9-b296-d2ac0d7efff5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:16.454845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22bda-ab5d-48c9-b296-d2ac0d7efff5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:16.455124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e22bda-ab5d-48c9-b296-d2ac0d7efff5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:16.456537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 169/345] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:16.463167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22bda-ab5d-48c9-b296-d2ac0d7efff5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:16.478368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 177/346] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:05:16.479841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e7cc0-3270-431c-883b-57f95e7e1285 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:16.480187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e7cc0-3270-431c-883b-57f95e7e1285 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:16.480516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e7cc0-3270-431c-883b-57f95e7e1285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:16.501018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-768e7cc0-3270-431c-883b-57f95e7e1285 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:16.502266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-768e7cc0-3270-431c-883b-57f95e7e1285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:16.592954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 178/347] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:16.593802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e7cc0-3270-431c-883b-57f95e7e1285 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:16.598020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:16.663222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e22bda-ab5d-48c9-b296-d2ac0d7efff5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:16.723613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b49d2-1f02-4409-8caa-fbf9711412b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:16.724495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b49d2-1f02-4409-8caa-fbf9711412b2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:16.724774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b49d2-1f02-4409-8caa-fbf9711412b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:16.725049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b49d2-1f02-4409-8caa-fbf9711412b2 None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:16.737574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e7cc0-3270-431c-883b-57f95e7e1285 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:16.740046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b49d2-1f02-4409-8caa-fbf9711412b2 None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:16.744416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b49d2-1f02-4409-8caa-fbf9711412b2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:16.748170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 170/348] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:16 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:16.750644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b49d2-1f02-4409-8caa-fbf9711412b2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:16.830086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:16.833414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2995ef-44a0-4235-ad21-c69658d48b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:16.834559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2995ef-44a0-4235-ad21-c69658d48b47 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:16.835070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2995ef-44a0-4235-ad21-c69658d48b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:16.835222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2995ef-44a0-4235-ad21-c69658d48b47 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:16.839304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2995ef-44a0-4235-ad21-c69658d48b47 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:16.840847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2995ef-44a0-4235-ad21-c69658d48b47 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:16.843196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 179/349] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:16 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:16.843961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2995ef-44a0-4235-ad21-c69658d48b47 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:16.893647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b49d2-1f02-4409-8caa-fbf9711412b2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:16.943775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b6043-f00e-4e76-8d0c-40fe732d5664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:16.944735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6043-f00e-4e76-8d0c-40fe732d5664 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:16.945144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6043-f00e-4e76-8d0c-40fe732d5664 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:16.945427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b6043-f00e-4e76-8d0c-40fe732d5664 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:16.958577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b6043-f00e-4e76-8d0c-40fe732d5664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f27b6043-f00e-4e76-8d0c-40fe732d5664 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:16.968583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:16.970986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:16.972932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:16.974443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 171/350] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:16.980269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b6043-f00e-4e76-8d0c-40fe732d5664 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:16.986511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:17.049004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2995ef-44a0-4235-ad21-c69658d48b47 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:17.109179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:17.110114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.110400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.110676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.111531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] RBAC: Adding query filter params (group.id=79b400bd2a62473fb5d1f485d779aefd, role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:17.111848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:17.113359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:17.128393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 180/351] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:16 2022] GET /identity/v3/role_assignments?group.id=79b400bd2a62473fb5d1f485d779aefd&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 260 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:17.131944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:17.210320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b6043-f00e-4e76-8d0c-40fe732d5664 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:17.226549 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:17.350834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ae616-11f5-4a1e-8aee-2df558aca46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:17.352069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ae616-11f5-4a1e-8aee-2df558aca46d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.352537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ae616-11f5-4a1e-8aee-2df558aca46d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.353003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040ae616-11f5-4a1e-8aee-2df558aca46d None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.355630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ae616-11f5-4a1e-8aee-2df558aca46d None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:17.358124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040ae616-11f5-4a1e-8aee-2df558aca46d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:17.361127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 172/352] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:16 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:17.368671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f203f-b849-4f46-b9a3-cae97c019025 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:17.368912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ae616-11f5-4a1e-8aee-2df558aca46d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:17.453340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440c032-89d1-4d0e-8bc2-69f6986631bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:17.454139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c032-89d1-4d0e-8bc2-69f6986631bc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.454421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c032-89d1-4d0e-8bc2-69f6986631bc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.454683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440c032-89d1-4d0e-8bc2-69f6986631bc None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.455491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440c032-89d1-4d0e-8bc2-69f6986631bc None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:17.455796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440c032-89d1-4d0e-8bc2-69f6986631bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:17.457214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440c032-89d1-4d0e-8bc2-69f6986631bc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:17.459734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:17.469560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 181/353] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:17.471111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2ad9-3753-443e-b1ec-d1c5526a6d47 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.471462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2ad9-3753-443e-b1ec-d1c5526a6d47 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.471802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2ad9-3753-443e-b1ec-d1c5526a6d47 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.473164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 182/354] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:17.474527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86dd33-59af-4d3e-b992-e34cb26ed784 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.474887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86dd33-59af-4d3e-b992-e34cb26ed784 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.475232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86dd33-59af-4d3e-b992-e34cb26ed784 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.476626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 183/355] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:17.487690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118899c-c60c-4e66-b044-830c9b356d06 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.487965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118899c-c60c-4e66-b044-830c9b356d06 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.488247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118899c-c60c-4e66-b044-830c9b356d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.511643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b118899c-c60c-4e66-b044-830c9b356d06 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:17.512829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b118899c-c60c-4e66-b044-830c9b356d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:17.533971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040ae616-11f5-4a1e-8aee-2df558aca46d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:17.571862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:17.572703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.572997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.573278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.574213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] RBAC: Adding query filter params (group.id=79b400bd2a62473fb5d1f485d779aefd, role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:17.574530 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:17.576010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:17.585745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 173/356] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:17 2022] GET /identity/v3/role_assignments?group.id=79b400bd2a62473fb5d1f485d779aefd&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:17.588140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:17.617900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 184/357] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:17.620154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f4a4-725a-44ae-bc49-f0491ea3c51c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.621927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f4a4-725a-44ae-bc49-f0491ea3c51c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.622451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f4a4-725a-44ae-bc49-f0491ea3c51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.682923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3681f4a4-725a-44ae-bc49-f0491ea3c51c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:17.684615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3681f4a4-725a-44ae-bc49-f0491ea3c51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:17.692207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:17.796721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 185/358] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:17.797942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b9341-3ca7-4fc2-a2ed-d9a9971c014b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.798232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b9341-3ca7-4fc2-a2ed-d9a9971c014b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.798509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b9341-3ca7-4fc2-a2ed-d9a9971c014b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.799930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 186/359] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:17.800479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b9341-3ca7-4fc2-a2ed-d9a9971c014b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:17.825044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3577c-60a3-437e-9ecc-184c9ca44e69 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:17.872363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9056f-7073-4c8d-a4d4-92811a8327a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:17.873048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9056f-7073-4c8d-a4d4-92811a8327a9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:17.873307 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9056f-7073-4c8d-a4d4-92811a8327a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:17.873617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f9056f-7073-4c8d-a4d4-92811a8327a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:17.888640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:17.955989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b9341-3ca7-4fc2-a2ed-d9a9971c014b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:17.970349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9056f-7073-4c8d-a4d4-92811a8327a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:17.971945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f9056f-7073-4c8d-a4d4-92811a8327a9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:18.019278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:18.022698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7765ec-b482-4f2e-91d5-1da98e796eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:18.024236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7765ec-b482-4f2e-91d5-1da98e796eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:18.024765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7765ec-b482-4f2e-91d5-1da98e796eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:18.026193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7765ec-b482-4f2e-91d5-1da98e796eb4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:18.041849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7765ec-b482-4f2e-91d5-1da98e796eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-cc7765ec-b482-4f2e-91d5-1da98e796eb4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:18.053273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:18.054296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 18 14:05:18.055503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:18.060097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 187/360] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:18.062666 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3095-9e78-4cf4-83f7-3d782856e853 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:18.063173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3095-9e78-4cf4-83f7-3d782856e853 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:18.063711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc3095-9e78-4cf4-83f7-3d782856e853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:18.090621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-87fc3095-9e78-4cf4-83f7-3d782856e853 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:18.094157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87fc3095-9e78-4cf4-83f7-3d782856e853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:18.158791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 174/361] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:18.159997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9056f-7073-4c8d-a4d4-92811a8327a9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:18.232819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 188/362] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:18.234097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc3095-9e78-4cf4-83f7-3d782856e853 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:18.311169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:18.327323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f9056f-7073-4c8d-a4d4-92811a8327a9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:18.387320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2d59c-157d-44eb-8bfe-7a84262bd3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:18.388370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2d59c-157d-44eb-8bfe-7a84262bd3fe None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:18.388761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2d59c-157d-44eb-8bfe-7a84262bd3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:18.389138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2d59c-157d-44eb-8bfe-7a84262bd3fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:18.392974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:18.405342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc3095-9e78-4cf4-83f7-3d782856e853 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:18.413349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2d59c-157d-44eb-8bfe-7a84262bd3fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1dc2d59c-157d-44eb-8bfe-7a84262bd3fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:18.424100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:18.425171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:18.426000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:18.429834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 175/363] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:05:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:18.430910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2d59c-157d-44eb-8bfe-7a84262bd3fe None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:18.459870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:18.460563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:18.460842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:18.461115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:18.461909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:18.462216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:18.464837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:18.486020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 189/364] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:18.487297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:18.584461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2d59c-157d-44eb-8bfe-7a84262bd3fe None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:18.625903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d3fa2-81e5-45c0-a252-37ac5286aecd None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:18.631425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d76033-be5e-493f-aa29-b7a438f375d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:18.632326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d76033-be5e-493f-aa29-b7a438f375d2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:18.632608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d76033-be5e-493f-aa29-b7a438f375d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:18.632887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d76033-be5e-493f-aa29-b7a438f375d2 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:18.634266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d76033-be5e-493f-aa29-b7a438f375d2 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:18.635841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d76033-be5e-493f-aa29-b7a438f375d2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:18.646164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 176/365] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:18 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:18.647114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d76033-be5e-493f-aa29-b7a438f375d2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:18.661161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:18.673249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb4216-893e-4ae9-859f-7e999858f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:18.674068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb4216-893e-4ae9-859f-7e999858f98d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:18.674354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb4216-893e-4ae9-859f-7e999858f98d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:18.674631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb4216-893e-4ae9-859f-7e999858f98d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:18.687416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb4216-893e-4ae9-859f-7e999858f98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-fafb4216-893e-4ae9-859f-7e999858f98d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:18.696831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:18.698669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:05:18.699588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:18.701674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 190/366] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:05:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:18.702784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb4216-893e-4ae9-859f-7e999858f98d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:18.742502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:18.792089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d76033-be5e-493f-aa29-b7a438f375d2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:18.823347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb4216-893e-4ae9-859f-7e999858f98d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:18.836497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:18.837205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:18.837610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:18.837917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:18.838845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:18.839151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:18.840730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:18.852780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 177/367] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:18.853513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:18.876177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab8623-68c2-4524-b45a-303435c9f7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:18.877898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab8623-68c2-4524-b45a-303435c9f7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:18.878409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab8623-68c2-4524-b45a-303435c9f7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:18.878868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab8623-68c2-4524-b45a-303435c9f7c7 None admin] PATH_INFO: `/v3/users/62cbe777d2244a22997a6d8372402472` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:18.880562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab8623-68c2-4524-b45a-303435c9f7c7 None admin] RBAC: Authorizing `identity:get_user(user_id=62cbe777d2244a22997a6d8372402472)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:18.882636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab8623-68c2-4524-b45a-303435c9f7c7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:18.885109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 191/368] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:05:18 2022] GET /identity/v3/users/62cbe777d2244a22997a6d8372402472 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:18.886445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab8623-68c2-4524-b45a-303435c9f7c7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:18.934369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:19.021702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab8623-68c2-4524-b45a-303435c9f7c7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:19.047178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275098f-1bcc-4a05-bcfc-83405501a3ae None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:19.068433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:19.069843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:19.070394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:19.070826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:19.071867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:19.072386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:19.074290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:19.086377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 192/369] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:19.088027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:19.092047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:19.106791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844a917-c698-418e-8e31-10f420520e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:19.108297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844a917-c698-418e-8e31-10f420520e0e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:19.109172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844a917-c698-418e-8e31-10f420520e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:19.109318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844a917-c698-418e-8e31-10f420520e0e None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:19.111708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844a917-c698-418e-8e31-10f420520e0e None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:19.116825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844a917-c698-418e-8e31-10f420520e0e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:19.119443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 178/370] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:18 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:19.120434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844a917-c698-418e-8e31-10f420520e0e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:19.220400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e24e64-bb78-4815-9da0-d793beb2884b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:19.224265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:19.266520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e1280-2291-4506-b771-b9f2097333f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:19.267453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e1280-2291-4506-b771-b9f2097333f3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:19.267875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e1280-2291-4506-b771-b9f2097333f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:19.268289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e1280-2291-4506-b771-b9f2097333f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:19.319987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844a917-c698-418e-8e31-10f420520e0e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:19.380924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e1280-2291-4506-b771-b9f2097333f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:19.383916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e1280-2291-4506-b771-b9f2097333f3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:19.389281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:19.390184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:19.390640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:19.391070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:19.392786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] RBAC: Adding query filter params (role.id=6e98f7101fb9430a87421182e7b6f4a8, user.id=62cbe777d2244a22997a6d8372402472, scope.system=all) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:19.393060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:19.395673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:19.419962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 179/371] 10.222.0.71 () {56 vars in 1308 bytes} [Mon Jul 18 14:05:19 2022] GET /identity/v3/role_assignments?user.id=62cbe777d2244a22997a6d8372402472&scope.system=all&role.id=6e98f7101fb9430a87421182e7b6f4a8 => generated 510 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:19.421406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:19.508144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:19.514788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 193/372] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:19.516161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e1280-2291-4506-b771-b9f2097333f3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:19.542939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:19.623322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf5311-cf8b-41fc-bb1e-813932fe3077 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:19.635616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e1280-2291-4506-b771-b9f2097333f3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:19.678417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:19.679464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:19.679977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:19.680463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:19.681502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] RBAC: Adding query filter params (group.id=79b400bd2a62473fb5d1f485d779aefd, role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=5c8b586142aa493484ff06e0a70778cf) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:19.682235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:19.684034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:19.688396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e1a3f-2282-4554-b233-9a9c9173b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:19.689382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e1a3f-2282-4554-b233-9a9c9173b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:19.689934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e1a3f-2282-4554-b233-9a9c9173b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:19.690283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e1a3f-2282-4554-b233-9a9c9173b1f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:19.692882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 194/373] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:19 2022] GET /identity/v3/role_assignments?group.id=79b400bd2a62473fb5d1f485d779aefd&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:19.694168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:19.767782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:19.829003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde96f8-2321-4784-a6c5-4c19961b473d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:19.833188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e1a3f-2282-4554-b233-9a9c9173b1f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:19.835735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831e1a3f-2282-4554-b233-9a9c9173b1f2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:19.872969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af4406-0ff3-4094-9568-c2bfe69e65d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:19.874324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af4406-0ff3-4094-9568-c2bfe69e65d8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:19.874861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af4406-0ff3-4094-9568-c2bfe69e65d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:19.875338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af4406-0ff3-4094-9568-c2bfe69e65d8 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:19.877044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af4406-0ff3-4094-9568-c2bfe69e65d8 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:19.878890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af4406-0ff3-4094-9568-c2bfe69e65d8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:19.881339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 195/374] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:19 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:19.890861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af4406-0ff3-4094-9568-c2bfe69e65d8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:19.957064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:20.035379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 180/375] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:20.044084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e1a3f-2282-4554-b233-9a9c9173b1f2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:20.058354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af4406-0ff3-4094-9568-c2bfe69e65d8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:20.111892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:20.125170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654a21e-14e9-4652-86fc-f1ff0cdaa948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:20.127482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654a21e-14e9-4652-86fc-f1ff0cdaa948 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:20.128166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654a21e-14e9-4652-86fc-f1ff0cdaa948 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:20.128834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654a21e-14e9-4652-86fc-f1ff0cdaa948 None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:20.131529 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654a21e-14e9-4652-86fc-f1ff0cdaa948 None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:20.135260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654a21e-14e9-4652-86fc-f1ff0cdaa948 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:20.139094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 196/376] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:19 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:20.147290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654a21e-14e9-4652-86fc-f1ff0cdaa948 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:20.191224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831e1a3f-2282-4554-b233-9a9c9173b1f2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:20.235527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a8fb-5bed-430b-88fe-c087b5cd053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:20.236993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a8fb-5bed-430b-88fe-c087b5cd053c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:20.237468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a8fb-5bed-430b-88fe-c087b5cd053c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:20.237945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a8fb-5bed-430b-88fe-c087b5cd053c None admin] PATH_INFO: `/v3/projects/c1db40e87aaf497fb0528cd53913ada9` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:20.239886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a8fb-5bed-430b-88fe-c087b5cd053c None admin] RBAC: Authorizing `identity:get_project(project_id=c1db40e87aaf497fb0528cd53913ada9)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:20.242440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a8fb-5bed-430b-88fe-c087b5cd053c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:20.246510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 181/377] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:20 2022] GET /identity/v3/projects/c1db40e87aaf497fb0528cd53913ada9 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:20.248440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01e6a-8141-4c14-aa86-f0985f640862 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:20.248718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01e6a-8141-4c14-aa86-f0985f640862 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:20.249028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e01e6a-8141-4c14-aa86-f0985f640862 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:20.250814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 182/378] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:20.251595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e01e6a-8141-4c14-aa86-f0985f640862 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:20.284246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:20.309523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654a21e-14e9-4652-86fc-f1ff0cdaa948 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:20.356211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e7b71-fa94-48ee-b5b8-a9f5d78e676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:20.356655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e7b71-fa94-48ee-b5b8-a9f5d78e676a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:20.357033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e7b71-fa94-48ee-b5b8-a9f5d78e676a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:20.357378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e7b71-fa94-48ee-b5b8-a9f5d78e676a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:20.358429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e7b71-fa94-48ee-b5b8-a9f5d78e676a None admin] RBAC: Adding query filter params (group.id=79b400bd2a62473fb5d1f485d779aefd, role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=5c8b586142aa493484ff06e0a70778cf) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:20.358968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e7b71-fa94-48ee-b5b8-a9f5d78e676a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:20.360472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e7b71-fa94-48ee-b5b8-a9f5d78e676a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:20.367699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 197/379] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:20 2022] GET /identity/v3/role_assignments?group.id=79b400bd2a62473fb5d1f485d779aefd&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:20.369874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3af2-4563-43ab-88da-a8cf075092d1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:20.370161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3af2-4563-43ab-88da-a8cf075092d1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:20.370431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3af2-4563-43ab-88da-a8cf075092d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:20.403752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-78cc3af2-4563-43ab-88da-a8cf075092d1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:20.404958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-78cc3af2-4563-43ab-88da-a8cf075092d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:20.442489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e01e6a-8141-4c14-aa86-f0985f640862 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:20.503942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f3af1-783a-4f63-8001-7b84c305facd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:20.505939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3af1-783a-4f63-8001-7b84c305facd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:20.507097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3af1-783a-4f63-8001-7b84c305facd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:20.507988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f3af1-783a-4f63-8001-7b84c305facd None admin] PATH_INFO: `/v3/groups/8fd181240db74a819704cdb3469a5cf4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:20.519515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f3af1-783a-4f63-8001-7b84c305facd None admin] RBAC: Authorizing `identity:get_group(group_id=8fd181240db74a819704cdb3469a5cf4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:20.523497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f3af1-783a-4f63-8001-7b84c305facd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:20.527466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 183/380] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:20 2022] GET /identity/v3/groups/8fd181240db74a819704cdb3469a5cf4 => generated 229 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:20.535166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f3af1-783a-4f63-8001-7b84c305facd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:20.576849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:20.628211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 198/381] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:20.675966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc3af2-4563-43ab-88da-a8cf075092d1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:20.718048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:20.745236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f3af1-783a-4f63-8001-7b84c305facd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:20.791546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:20.792755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:20.793192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:20.793589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:20.794722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] RBAC: Adding query filter params (group.id=8fd181240db74a819704cdb3469a5cf4, role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=c1db40e87aaf497fb0528cd53913ada9) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:20.795048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:20.797249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:20.805953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 184/382] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:20 2022] GET /identity/v3/role_assignments?group.id=8fd181240db74a819704cdb3469a5cf4&scope.project.id=c1db40e87aaf497fb0528cd53913ada9&role.id=5c672a1b2979444383c587a215adeb35 => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:20.857605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc3af2-4563-43ab-88da-a8cf075092d1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:20.902697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb9dc-2b0f-470d-8676-003f92c637d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:20.903936 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb9dc-2b0f-470d-8676-003f92c637d0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:20.904369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb9dc-2b0f-470d-8676-003f92c637d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:20.904896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb9dc-2b0f-470d-8676-003f92c637d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:20.934535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb9dc-2b0f-470d-8676-003f92c637d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ba5fb9dc-2b0f-470d-8676-003f92c637d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:20.942670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:20.943654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:20.944449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 199/383] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:20.950708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:21.088793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567b1c3-0365-48e3-96a8-a8259f3f4dd5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:21.128987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cfc6f8-2520-492a-a7ed-7c04de8ab3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:21.129708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfc6f8-2520-492a-a7ed-7c04de8ab3aa None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:21.130006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfc6f8-2520-492a-a7ed-7c04de8ab3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:21.130279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cfc6f8-2520-492a-a7ed-7c04de8ab3aa None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:21.131043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfc6f8-2520-492a-a7ed-7c04de8ab3aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:21.131352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfc6f8-2520-492a-a7ed-7c04de8ab3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:21.132810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cfc6f8-2520-492a-a7ed-7c04de8ab3aa None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:21.144749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 185/384] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:21.150864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb9dc-2b0f-470d-8676-003f92c637d0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:21.212385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278f12b-d3f9-4593-9b7c-881b90cbc22b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:21.213107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278f12b-d3f9-4593-9b7c-881b90cbc22b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:21.213107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278f12b-d3f9-4593-9b7c-881b90cbc22b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:21.214923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 186/385] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:21.220639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01e32-e55c-4505-9f81-481a759fb40e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:21.221054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01e32-e55c-4505-9f81-481a759fb40e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:21.221219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c01e32-e55c-4505-9f81-481a759fb40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:21.241677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-10c01e32-e55c-4505-9f81-481a759fb40e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:21.242889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-10c01e32-e55c-4505-9f81-481a759fb40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:21.251428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:21.254249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:21.382602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb9dc-2b0f-470d-8676-003f92c637d0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:21.397367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 187/386] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:21.433871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01e32-e55c-4505-9f81-481a759fb40e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:21.456063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483bfbe-86ed-4552-bc0e-7b9372da11ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:21.457599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483bfbe-86ed-4552-bc0e-7b9372da11ad None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:21.462760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483bfbe-86ed-4552-bc0e-7b9372da11ad None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:21.464522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483bfbe-86ed-4552-bc0e-7b9372da11ad None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:21.473229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483bfbe-86ed-4552-bc0e-7b9372da11ad None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:21.476753 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483bfbe-86ed-4552-bc0e-7b9372da11ad None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:21.484736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 200/387] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:21 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:21.492086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483bfbe-86ed-4552-bc0e-7b9372da11ad None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:21.588235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c01e32-e55c-4505-9f81-481a759fb40e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:21.654249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95238f82-5a82-4a7a-b6bc-140337074c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:21.655678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95238f82-5a82-4a7a-b6bc-140337074c5b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:21.656170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95238f82-5a82-4a7a-b6bc-140337074c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:21.656663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95238f82-5a82-4a7a-b6bc-140337074c5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:21.694553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:21.714429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95238f82-5a82-4a7a-b6bc-140337074c5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-95238f82-5a82-4a7a-b6bc-140337074c5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:21.723237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:21.724175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:21.724966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:21.727765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 188/388] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:05:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:21.754686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95238f82-5a82-4a7a-b6bc-140337074c5b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:21.817312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483bfbe-86ed-4552-bc0e-7b9372da11ad None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:21.845712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:21.870913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793eb4d-ac72-4bb3-bc3d-bdfbae3fe719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:21.872977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793eb4d-ac72-4bb3-bc3d-bdfbae3fe719 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:21.873607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793eb4d-ac72-4bb3-bc3d-bdfbae3fe719 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:21.874347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793eb4d-ac72-4bb3-bc3d-bdfbae3fe719 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:21.876839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793eb4d-ac72-4bb3-bc3d-bdfbae3fe719 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:21.879702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793eb4d-ac72-4bb3-bc3d-bdfbae3fe719 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:21.882089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 201/389] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:21 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:21.888165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793eb4d-ac72-4bb3-bc3d-bdfbae3fe719 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:21.961677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95238f82-5a82-4a7a-b6bc-140337074c5b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:22.036123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:22.037236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:22.037783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:22.038268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:22.039705 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:22.040251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:22.043052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:22.080099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 189/390] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:22.088917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:22.147266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793eb4d-ac72-4bb3-bc3d-bdfbae3fe719 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:22.177410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:22.229403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955795f1-0e59-4114-b94b-0e6d73c5d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:22.229992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955795f1-0e59-4114-b94b-0e6d73c5d68b None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:22.230233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955795f1-0e59-4114-b94b-0e6d73c5d68b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:22.231502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955795f1-0e59-4114-b94b-0e6d73c5d68b None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/groups/79b400bd2a62473fb5d1f485d779aefd/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:22.236652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955795f1-0e59-4114-b94b-0e6d73c5d68b None admin] RBAC: Authorizing `identity:create_grant(project_id=43134d25fcbf4ee6a4e0b977b35eddeb, group_id=79b400bd2a62473fb5d1f485d779aefd, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:22.238176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955795f1-0e59-4114-b94b-0e6d73c5d68b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:22.264919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 202/391] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 18 14:05:21 2022] PUT /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/groups/79b400bd2a62473fb5d1f485d779aefd/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:05:22.274482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b203-1ea2-4d23-8f6d-f291b62e250f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:22.325874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79701c6-3593-44f5-bbcd-e7e6972f66f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:22.326775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79701c6-3593-44f5-bbcd-e7e6972f66f1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:22.327057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79701c6-3593-44f5-bbcd-e7e6972f66f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:22.327331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79701c6-3593-44f5-bbcd-e7e6972f66f1 None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:22.334709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79701c6-3593-44f5-bbcd-e7e6972f66f1 None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:22.336249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79701c6-3593-44f5-bbcd-e7e6972f66f1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:22.338583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 190/392] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:22 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:22.344930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955795f1-0e59-4114-b94b-0e6d73c5d68b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:22.428010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:22.513740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955795f1-0e59-4114-b94b-0e6d73c5d68b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:22.575585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb04f4-6e4b-4235-8222-a8ac3fd4d227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:22.576490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb04f4-6e4b-4235-8222-a8ac3fd4d227 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:22.576776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb04f4-6e4b-4235-8222-a8ac3fd4d227 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:22.577045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb04f4-6e4b-4235-8222-a8ac3fd4d227 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:22.578359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb04f4-6e4b-4235-8222-a8ac3fd4d227 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:22.579938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb04f4-6e4b-4235-8222-a8ac3fd4d227 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:22.583342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 203/393] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:22 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:22.591118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79701c6-3593-44f5-bbcd-e7e6972f66f1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:22.764726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79701c6-3593-44f5-bbcd-e7e6972f66f1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:22.802657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:22.810701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1762d-2c65-4992-b5f1-271a2fe260cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:22.811740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1762d-2c65-4992-b5f1-271a2fe260cc None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:22.812201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1762d-2c65-4992-b5f1-271a2fe260cc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:22.812489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1762d-2c65-4992-b5f1-271a2fe260cc None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/groups/79b400bd2a62473fb5d1f485d779aefd/roles/89d5f9aaa6284371b4d5205fea92cba1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:22.819390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1762d-2c65-4992-b5f1-271a2fe260cc None admin] RBAC: Authorizing `identity:create_grant(project_id=43134d25fcbf4ee6a4e0b977b35eddeb, group_id=79b400bd2a62473fb5d1f485d779aefd, role_id=89d5f9aaa6284371b4d5205fea92cba1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:22.820895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1762d-2c65-4992-b5f1-271a2fe260cc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:22.839840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 191/394] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 18 14:05:22 2022] PUT /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb/groups/79b400bd2a62473fb5d1f485d779aefd/roles/89d5f9aaa6284371b4d5205fea92cba1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:22.898031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78234f-4560-4cfe-9be2-e29d883922d8 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:22.898542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78234f-4560-4cfe-9be2-e29d883922d8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:22.899117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba78234f-4560-4cfe-9be2-e29d883922d8 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:22.901602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 204/395] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:22.908841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42435d0e-107f-459a-8818-05e24c9e418c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:22.909256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42435d0e-107f-459a-8818-05e24c9e418c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:22.909683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42435d0e-107f-459a-8818-05e24c9e418c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:22.933839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-42435d0e-107f-459a-8818-05e24c9e418c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:22.935114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-42435d0e-107f-459a-8818-05e24c9e418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:23.035912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 192/396] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:23.079589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78234f-4560-4cfe-9be2-e29d883922d8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:23.246926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba78234f-4560-4cfe-9be2-e29d883922d8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:23.275141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:23.302063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084854bf-9724-4c7f-aa1a-1893de77d002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:23.302867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084854bf-9724-4c7f-aa1a-1893de77d002 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:23.303152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084854bf-9724-4c7f-aa1a-1893de77d002 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:23.303497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084854bf-9724-4c7f-aa1a-1893de77d002 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:23.335326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084854bf-9724-4c7f-aa1a-1893de77d002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-084854bf-9724-4c7f-aa1a-1893de77d002 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:23.345288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:23.347073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:23.349673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 205/397] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:23.359899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42435d0e-107f-459a-8818-05e24c9e418c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:23.471313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677abce-cf3e-4a64-b5e5-e5eed30c4efe None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:23.471768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677abce-cf3e-4a64-b5e5-e5eed30c4efe None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:23.471998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677abce-cf3e-4a64-b5e5-e5eed30c4efe None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:23.478097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 206/398] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:23.481849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc6bf5-a2e5-4df1-9350-d89d29afc7bb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:23.482316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc6bf5-a2e5-4df1-9350-d89d29afc7bb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:23.482610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc6bf5-a2e5-4df1-9350-d89d29afc7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:23.505597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-53bc6bf5-a2e5-4df1-9350-d89d29afc7bb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:23.507689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-53bc6bf5-a2e5-4df1-9350-d89d29afc7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:23.515071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42435d0e-107f-459a-8818-05e24c9e418c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:23.553733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:23.585875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:23.587015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:23.587508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:23.588027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:23.589458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] RBAC: Adding query filter params (name=member) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:23.591597 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:23.594182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:23.639496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 193/399] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:23.651025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:23.702876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 207/400] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:23.779363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc6bf5-a2e5-4df1-9350-d89d29afc7bb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:23.874256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458d5ed-f48a-4012-b46c-84aa8fa2284d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:23.925735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b6add-2b6a-4b8d-9425-cf8a47442404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:23.926657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b6add-2b6a-4b8d-9425-cf8a47442404 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:23.926946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b6add-2b6a-4b8d-9425-cf8a47442404 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:23.927229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b6add-2b6a-4b8d-9425-cf8a47442404 None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:23.928827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b6add-2b6a-4b8d-9425-cf8a47442404 None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:23.931454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b6add-2b6a-4b8d-9425-cf8a47442404 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:23.933788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 194/401] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:23 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:23.946287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b6add-2b6a-4b8d-9425-cf8a47442404 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:24.076249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc6bf5-a2e5-4df1-9350-d89d29afc7bb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:24.090354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:24.091807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:24.181174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a568-34d1-4589-8022-076da10b536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:24.182241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a568-34d1-4589-8022-076da10b536b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:24.182512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a568-34d1-4589-8022-076da10b536b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:24.182793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8a568-34d1-4589-8022-076da10b536b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:24.215312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8a568-34d1-4589-8022-076da10b536b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:24.220602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b6add-2b6a-4b8d-9425-cf8a47442404 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-81f8a568-34d1-4589-8022-076da10b536b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:24.234248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:24.236024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:24.237625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:24.242428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 208/402] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:05:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:24.263796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a568-34d1-4589-8022-076da10b536b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:24.280609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf943b-922c-4d78-9302-a7583ae67f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:24.281566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf943b-922c-4d78-9302-a7583ae67f38 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:24.281877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf943b-922c-4d78-9302-a7583ae67f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:24.282161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf943b-922c-4d78-9302-a7583ae67f38 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:24.283568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf943b-922c-4d78-9302-a7583ae67f38 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:24.285055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf943b-922c-4d78-9302-a7583ae67f38 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:24.287099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 195/403] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:23 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:24.293240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf943b-922c-4d78-9302-a7583ae67f38 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:24.426481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8a568-34d1-4589-8022-076da10b536b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:24.468208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93313b06-a933-4cc0-8ad1-8965f78f25bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:24.469367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93313b06-a933-4cc0-8ad1-8965f78f25bb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:24.469904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93313b06-a933-4cc0-8ad1-8965f78f25bb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:24.470384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93313b06-a933-4cc0-8ad1-8965f78f25bb None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:24.471457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93313b06-a933-4cc0-8ad1-8965f78f25bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:24.472067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93313b06-a933-4cc0-8ad1-8965f78f25bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:24.473702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93313b06-a933-4cc0-8ad1-8965f78f25bb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:24.477837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf943b-922c-4d78-9302-a7583ae67f38 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:24.485104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 209/404] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:24.486934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495af4a-5e23-4c88-96eb-e2f6feb487a7 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:24.487426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495af4a-5e23-4c88-96eb-e2f6feb487a7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:24.487959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1495af4a-5e23-4c88-96eb-e2f6feb487a7 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:24.489610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 210/405] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:24.495496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495af4a-5e23-4c88-96eb-e2f6feb487a7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:24.502540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:24.529685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecaa18d-8d89-4006-b7f7-a42463427241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:24.531267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaa18d-8d89-4006-b7f7-a42463427241 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:24.531764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaa18d-8d89-4006-b7f7-a42463427241 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:24.532262 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecaa18d-8d89-4006-b7f7-a42463427241 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf/groups/79b400bd2a62473fb5d1f485d779aefd/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:24.534532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:24.536108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecaa18d-8d89-4006-b7f7-a42463427241 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8b586142aa493484ff06e0a70778cf, group_id=79b400bd2a62473fb5d1f485d779aefd, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:24.540016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecaa18d-8d89-4006-b7f7-a42463427241 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:24.574683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 196/406] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 18 14:05:24 2022] PUT /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf/groups/79b400bd2a62473fb5d1f485d779aefd/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:05:24.576595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed07930-5137-48f9-927e-e53910a32b12 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:24.577193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed07930-5137-48f9-927e-e53910a32b12 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:24.577775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed07930-5137-48f9-927e-e53910a32b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:24.621472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bed07930-5137-48f9-927e-e53910a32b12 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:24.626158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bed07930-5137-48f9-927e-e53910a32b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:24.673841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1495af4a-5e23-4c88-96eb-e2f6feb487a7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:24.741924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2906a4-db80-4487-aafa-f4b679014be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:24.744553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2906a4-db80-4487-aafa-f4b679014be0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:24.745343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2906a4-db80-4487-aafa-f4b679014be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:24.746093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2906a4-db80-4487-aafa-f4b679014be0 None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:24.748851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2906a4-db80-4487-aafa-f4b679014be0 None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:24.752684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2906a4-db80-4487-aafa-f4b679014be0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:24.754496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 197/407] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:24.757347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 211/408] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:24 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:24.766030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed07930-5137-48f9-927e-e53910a32b12 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:24.820199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2906a4-db80-4487-aafa-f4b679014be0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:24.904306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed07930-5137-48f9-927e-e53910a32b12 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:24.934098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:24.946129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630855c8-3c6e-45ee-b3bd-d9e957bd9f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:24.947078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630855c8-3c6e-45ee-b3bd-d9e957bd9f6f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:24.947439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630855c8-3c6e-45ee-b3bd-d9e957bd9f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:24.947846 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630855c8-3c6e-45ee-b3bd-d9e957bd9f6f None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:24.950076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630855c8-3c6e-45ee-b3bd-d9e957bd9f6f None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:24.952107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630855c8-3c6e-45ee-b3bd-d9e957bd9f6f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:24.954666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 198/409] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:24 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:24.961141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630855c8-3c6e-45ee-b3bd-d9e957bd9f6f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:25.052448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:25.060706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2906a4-db80-4487-aafa-f4b679014be0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:25.145296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630855c8-3c6e-45ee-b3bd-d9e957bd9f6f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:25.155244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a5d19-4530-4b2a-a727-ea8f3bacb520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:25.156717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a5d19-4530-4b2a-a727-ea8f3bacb520 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:25.157219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a5d19-4530-4b2a-a727-ea8f3bacb520 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:25.158315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a5d19-4530-4b2a-a727-ea8f3bacb520 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:25.210069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a5d19-4530-4b2a-a727-ea8f3bacb520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9f3a5d19-4530-4b2a-a727-ea8f3bacb520 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:25.242885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:25.244834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:05:25.246287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:25.253464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 212/410] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:05:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:25.264719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a5d19-4530-4b2a-a727-ea8f3bacb520 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:25.302377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f647b67-74dd-4e92-a728-bdda4c2992c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:25.305383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f647b67-74dd-4e92-a728-bdda4c2992c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:25.305811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f647b67-74dd-4e92-a728-bdda4c2992c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:25.309400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f647b67-74dd-4e92-a728-bdda4c2992c7 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf/groups/79b400bd2a62473fb5d1f485d779aefd/roles/89d5f9aaa6284371b4d5205fea92cba1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:25.318342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f647b67-74dd-4e92-a728-bdda4c2992c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c8b586142aa493484ff06e0a70778cf, group_id=79b400bd2a62473fb5d1f485d779aefd, role_id=89d5f9aaa6284371b4d5205fea92cba1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:25.327344 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f647b67-74dd-4e92-a728-bdda4c2992c7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:25.364076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 199/411] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 18 14:05:24 2022] PUT /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf/groups/79b400bd2a62473fb5d1f485d779aefd/roles/89d5f9aaa6284371b4d5205fea92cba1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:05:25.547677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a5d19-4530-4b2a-a727-ea8f3bacb520 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:25.619689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fdb04-6414-455c-bf6a-08e223fa0e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:25.621387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdb04-6414-455c-bf6a-08e223fa0e45 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:25.622266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdb04-6414-455c-bf6a-08e223fa0e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:25.623118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fdb04-6414-455c-bf6a-08e223fa0e45 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:25.624893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fdb04-6414-455c-bf6a-08e223fa0e45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:25.625629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fdb04-6414-455c-bf6a-08e223fa0e45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:25.628810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fdb04-6414-455c-bf6a-08e223fa0e45 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:25.651247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 213/412] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:25.657555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f647b67-74dd-4e92-a728-bdda4c2992c7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:25.715236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d69a-b366-4f87-a4e2-983c496bb039 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:25.715654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d69a-b366-4f87-a4e2-983c496bb039 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:25.716052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4d69a-b366-4f87-a4e2-983c496bb039 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:25.716598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:25.718505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 214/413] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:25.727000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659295cf-b416-40bc-9800-0607cdade7bd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:25.727624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659295cf-b416-40bc-9800-0607cdade7bd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:25.728162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659295cf-b416-40bc-9800-0607cdade7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:25.751551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-659295cf-b416-40bc-9800-0607cdade7bd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:25.753010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-659295cf-b416-40bc-9800-0607cdade7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:25.790402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:25.870800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f647b67-74dd-4e92-a728-bdda4c2992c7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:25.893074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 215/414] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:25.918003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659295cf-b416-40bc-9800-0607cdade7bd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:25.969600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72cf8d-571c-46fd-ad0f-611b3e19df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:25.970518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72cf8d-571c-46fd-ad0f-611b3e19df1d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:25.970799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72cf8d-571c-46fd-ad0f-611b3e19df1d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:25.971071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72cf8d-571c-46fd-ad0f-611b3e19df1d None admin] PATH_INFO: `/v3/groups/8fd181240db74a819704cdb3469a5cf4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:25.973175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72cf8d-571c-46fd-ad0f-611b3e19df1d None admin] RBAC: Authorizing `identity:get_group(group_id=8fd181240db74a819704cdb3469a5cf4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:25.974679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72cf8d-571c-46fd-ad0f-611b3e19df1d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:25.976415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 200/415] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:25 2022] GET /identity/v3/groups/8fd181240db74a819704cdb3469a5cf4 => generated 229 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:25.984131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72cf8d-571c-46fd-ad0f-611b3e19df1d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:26.069686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659295cf-b416-40bc-9800-0607cdade7bd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:26.110825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ee411-8272-463b-a0ed-36afa1161b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:26.112529 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ee411-8272-463b-a0ed-36afa1161b24 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.113224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ee411-8272-463b-a0ed-36afa1161b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.113880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ee411-8272-463b-a0ed-36afa1161b24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.115695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72cf8d-571c-46fd-ad0f-611b3e19df1d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:26.118733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:26.128713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ee411-8272-463b-a0ed-36afa1161b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f30ee411-8272-463b-a0ed-36afa1161b24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:26.137287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:26.138435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:26.139242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:26.141955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 216/416] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:26.143619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee02a2-e628-4c91-9229-52c4196ff0a0 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.144135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee02a2-e628-4c91-9229-52c4196ff0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.144638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee02a2-e628-4c91-9229-52c4196ff0a0 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.149736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 217/417] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:26.150981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee02a2-e628-4c91-9229-52c4196ff0a0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:26.161845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9c083-3e5a-472d-ad9b-d4ac2dc52957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:26.162734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9c083-3e5a-472d-ad9b-d4ac2dc52957 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.163041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9c083-3e5a-472d-ad9b-d4ac2dc52957 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.163428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9c083-3e5a-472d-ad9b-d4ac2dc52957 None admin] PATH_INFO: `/v3/projects/c1db40e87aaf497fb0528cd53913ada9` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.164696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9c083-3e5a-472d-ad9b-d4ac2dc52957 None admin] RBAC: Authorizing `identity:get_project(project_id=c1db40e87aaf497fb0528cd53913ada9)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:26.166546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:26.166751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9c083-3e5a-472d-ad9b-d4ac2dc52957 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:26.169478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 201/418] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:25 2022] GET /identity/v3/projects/c1db40e87aaf497fb0528cd53913ada9 => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:26.171539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775287-e605-408c-abd6-ebc33bc0e775 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.171834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775287-e605-408c-abd6-ebc33bc0e775 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.172127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775287-e605-408c-abd6-ebc33bc0e775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.191538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-70775287-e605-408c-abd6-ebc33bc0e775 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:26.192921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-70775287-e605-408c-abd6-ebc33bc0e775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:26.274517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 202/419] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:26.275765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70775287-e605-408c-abd6-ebc33bc0e775 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:26.306268 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee02a2-e628-4c91-9229-52c4196ff0a0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:26.363285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:26.364084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.364371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.364675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.366216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:26.366593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:26.368851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:26.384040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 218/420] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:26.384949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:26.430295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70775287-e605-408c-abd6-ebc33bc0e775 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:26.433356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:26.481913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6668e-4911-4b8e-8aaa-f8ed1fbb8e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:26.482908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6668e-4911-4b8e-8aaa-f8ed1fbb8e3e None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.483186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6668e-4911-4b8e-8aaa-f8ed1fbb8e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.483454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6668e-4911-4b8e-8aaa-f8ed1fbb8e3e None admin] PATH_INFO: `/v3/projects/c1db40e87aaf497fb0528cd53913ada9/groups/8fd181240db74a819704cdb3469a5cf4/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.485594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6668e-4911-4b8e-8aaa-f8ed1fbb8e3e None admin] RBAC: Authorizing `identity:create_grant(project_id=c1db40e87aaf497fb0528cd53913ada9, group_id=8fd181240db74a819704cdb3469a5cf4, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:26.487074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6668e-4911-4b8e-8aaa-f8ed1fbb8e3e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:26.487750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:26.506389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 203/421] 10.222.0.71 () {58 vars in 1442 bytes} [Mon Jul 18 14:05:26 2022] PUT /identity/v3/projects/c1db40e87aaf497fb0528cd53913ada9/groups/8fd181240db74a819704cdb3469a5cf4/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:26.507862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6668e-4911-4b8e-8aaa-f8ed1fbb8e3e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:26.532077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858f771-7c7b-41df-8d75-c276d0326d18 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:26.577380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3176da-8876-4b57-b65d-a554068e8d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:26.578298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3176da-8876-4b57-b65d-a554068e8d0f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.578794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3176da-8876-4b57-b65d-a554068e8d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.578974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3176da-8876-4b57-b65d-a554068e8d0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.591069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3176da-8876-4b57-b65d-a554068e8d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-cb3176da-8876-4b57-b65d-a554068e8d0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:26.600167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:26.602284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:26.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 219/422] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:05:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:26.608141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3176da-8876-4b57-b65d-a554068e8d0f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:26.680904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6668e-4911-4b8e-8aaa-f8ed1fbb8e3e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:26.750793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3176da-8876-4b57-b65d-a554068e8d0f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:26.753768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6152d-fb31-444d-9014-af534136dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:26.754691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6152d-fb31-444d-9014-af534136dc61 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.755171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6152d-fb31-444d-9014-af534136dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.755635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6152d-fb31-444d-9014-af534136dc61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.788205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:26.794527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:26.795190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:26.796049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:26.796660 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:26.797970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:26.798587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:26.800294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:26.813281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 220/423] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:26.821721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:26.850749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:26.913062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6152d-fb31-444d-9014-af534136dc61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:26.915568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc6152d-fb31-444d-9014-af534136dc61 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:26.990333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24653b13-c7e0-4ba8-8e0e-93fe7834c31d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:27.045495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5ff17-cd07-49a0-aee2-270c55a82951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:27.046361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff17-cd07-49a0-aee2-270c55a82951 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:27.046649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff17-cd07-49a0-aee2-270c55a82951 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:27.046917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5ff17-cd07-49a0-aee2-270c55a82951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:27.164378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 204/424] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:27.169978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6152d-fb31-444d-9014-af534136dc61 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:27.215960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:27.250309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:27.258897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5ff17-cd07-49a0-aee2-270c55a82951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:27.261197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5ff17-cd07-49a0-aee2-270c55a82951 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:27.297049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6152d-fb31-444d-9014-af534136dc61 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:27.352881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9945a-b55a-4aab-86cd-51d813270bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:27.354486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9945a-b55a-4aab-86cd-51d813270bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:27.354771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9945a-b55a-4aab-86cd-51d813270bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:27.355067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9945a-b55a-4aab-86cd-51d813270bd7 None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:27.361065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9945a-b55a-4aab-86cd-51d813270bd7 None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:27.362849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9945a-b55a-4aab-86cd-51d813270bd7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:27.368001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 205/425] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:27 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:27.371747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efc6e7-b7aa-4d53-bfb1-f83a50423340 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:27.372260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efc6e7-b7aa-4d53-bfb1-f83a50423340 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:27.372749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efc6e7-b7aa-4d53-bfb1-f83a50423340 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:27.375349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 206/426] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:27.377518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efc6e7-b7aa-4d53-bfb1-f83a50423340 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:27.446760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 221/427] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:27.448723 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db1a03-42a8-48e6-9e1e-df3b53ccaf5f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:27.449071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db1a03-42a8-48e6-9e1e-df3b53ccaf5f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:27.449419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db1a03-42a8-48e6-9e1e-df3b53ccaf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:27.474269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d9db1a03-42a8-48e6-9e1e-df3b53ccaf5f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:27.477046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d9db1a03-42a8-48e6-9e1e-df3b53ccaf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:27.533203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efc6e7-b7aa-4d53-bfb1-f83a50423340 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:27.581404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:27.592284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726d2e1-82bb-41b4-82e4-87ae20b1f8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:27.594017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726d2e1-82bb-41b4-82e4-87ae20b1f8ec None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:27.594550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726d2e1-82bb-41b4-82e4-87ae20b1f8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:27.595020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726d2e1-82bb-41b4-82e4-87ae20b1f8ec None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:27.596863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726d2e1-82bb-41b4-82e4-87ae20b1f8ec None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:27.599474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726d2e1-82bb-41b4-82e4-87ae20b1f8ec None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:27.602476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 207/428] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:27 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:27.604046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726d2e1-82bb-41b4-82e4-87ae20b1f8ec None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:27.635653 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 222/429] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:27.637293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db1a03-42a8-48e6-9e1e-df3b53ccaf5f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:27.671110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:27.735675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726d2e1-82bb-41b4-82e4-87ae20b1f8ec None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:27.763422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db1a03-42a8-48e6-9e1e-df3b53ccaf5f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:27.784664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8026-f35c-4a49-b3e2-44a11aa0352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:27.786244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8026-f35c-4a49-b3e2-44a11aa0352a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:27.786742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8026-f35c-4a49-b3e2-44a11aa0352a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:27.787241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc8026-f35c-4a49-b3e2-44a11aa0352a None admin] PATH_INFO: `/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:27.789524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc8026-f35c-4a49-b3e2-44a11aa0352a None admin] RBAC: Authorizing `identity:get_project(project_id=43134d25fcbf4ee6a4e0b977b35eddeb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:27.792524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc8026-f35c-4a49-b3e2-44a11aa0352a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:27.796376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 208/430] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:27 2022] GET /identity/v3/projects/43134d25fcbf4ee6a4e0b977b35eddeb => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:27.798194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8026-f35c-4a49-b3e2-44a11aa0352a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:27.817832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:27.818957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:27.819911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:27.820385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:27.821455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] RBAC: Adding query filter params (group.id=79b400bd2a62473fb5d1f485d779aefd, role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:27.822313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:27.825724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:27.834305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 223/431] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:27 2022] GET /identity/v3/role_assignments?group.id=79b400bd2a62473fb5d1f485d779aefd&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:27.835528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:27.973209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:27.979865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:27.999765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8ead5-f7ec-46d2-864f-b265ac6fb72e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:28.046223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb4bfb-eec3-44be-b8bb-13f46de1d1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:28.048110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb4bfb-eec3-44be-b8bb-13f46de1d1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.048640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb4bfb-eec3-44be-b8bb-13f46de1d1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.049156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb4bfb-eec3-44be-b8bb-13f46de1d1b1 None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.050866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb4bfb-eec3-44be-b8bb-13f46de1d1b1 None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:28.052931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb4bfb-eec3-44be-b8bb-13f46de1d1b1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:28.055960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 224/432] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:27 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:28.062841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb4bfb-eec3-44be-b8bb-13f46de1d1b1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:28.091780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc8026-f35c-4a49-b3e2-44a11aa0352a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:28.141141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6367ec28-1074-4716-839c-ebbf7da027c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:28.142036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ec28-1074-4716-839c-ebbf7da027c2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.142321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ec28-1074-4716-839c-ebbf7da027c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.142599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6367ec28-1074-4716-839c-ebbf7da027c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.158113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6367ec28-1074-4716-839c-ebbf7da027c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6367ec28-1074-4716-839c-ebbf7da027c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:28.167620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:28.168570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:28.169445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 209/433] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:28.170672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a4e25-934c-4414-8300-00127cdd7374 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.170778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a4e25-934c-4414-8300-00127cdd7374 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.171004 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a4e25-934c-4414-8300-00127cdd7374 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.172809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 210/434] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:28.178837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660775-900a-4a05-bbb0-ff008d46bbbe None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.179193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660775-900a-4a05-bbb0-ff008d46bbbe None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.179406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5660775-900a-4a05-bbb0-ff008d46bbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.205268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b5660775-900a-4a05-bbb0-ff008d46bbbe None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:28.207484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b5660775-900a-4a05-bbb0-ff008d46bbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:28.214322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb4bfb-eec3-44be-b8bb-13f46de1d1b1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:28.255278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:28.255879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.256158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.256446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.257274 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] RBAC: Adding query filter params (group.id=79b400bd2a62473fb5d1f485d779aefd, role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:28.257680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:28.260379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:28.267030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 225/435] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:28 2022] GET /identity/v3/role_assignments?group.id=79b400bd2a62473fb5d1f485d779aefd&scope.project.id=43134d25fcbf4ee6a4e0b977b35eddeb&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:28.267931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:28.294657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:28.362605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 211/436] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:28.388917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5660775-900a-4a05-bbb0-ff008d46bbbe None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:28.423610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a7c89-1da5-4ad2-a9aa-78b0f564cbfc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:28.480114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:28.481136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.481506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.482864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.483718 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] RBAC: Adding query filter params (name=member) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:28.484029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:28.485510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:28.499940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 226/437] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:05:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:28.506731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:28.507197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:28.600758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5660775-900a-4a05-bbb0-ff008d46bbbe None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:28.669802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:28.673148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668c25b-8e9c-4deb-a6ad-56b97e18dd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:28.674333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668c25b-8e9c-4deb-a6ad-56b97e18dd86 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.674648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668c25b-8e9c-4deb-a6ad-56b97e18dd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.675266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668c25b-8e9c-4deb-a6ad-56b97e18dd86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.699705 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5668c25b-8e9c-4deb-a6ad-56b97e18dd86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:28.712971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fd63c-472f-4e1c-bae0-762cc1d63498 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-5668c25b-8e9c-4deb-a6ad-56b97e18dd86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:28.713867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:28.714815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:28.715661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 212/438] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:05:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:28.721274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668c25b-8e9c-4deb-a6ad-56b97e18dd86 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:28.773473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6fbd9-f1cc-4b02-8f8c-224e1f7688ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:28.774817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6fbd9-f1cc-4b02-8f8c-224e1f7688ab None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.775399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6fbd9-f1cc-4b02-8f8c-224e1f7688ab None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.776378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6fbd9-f1cc-4b02-8f8c-224e1f7688ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.863152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668c25b-8e9c-4deb-a6ad-56b97e18dd86 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:28.897702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:28.930430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6fbd9-f1cc-4b02-8f8c-224e1f7688ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:28.934320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6fbd9-f1cc-4b02-8f8c-224e1f7688ab None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:28.936924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:28.938272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:28.938776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:28.940186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:28.941189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:28.941503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:28.950635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:28.967548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 213/439] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:28.975436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:29.059993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:29.100058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 227/440] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:05:29.109899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6fbd9-f1cc-4b02-8f8c-224e1f7688ab None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:29.134977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48849a-1db7-4062-af51-c3c75358b01d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:29.187801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaeae7f-b0af-4f04-8f6e-ab5fece36311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:29.188514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeae7f-b0af-4f04-8f6e-ab5fece36311 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.188802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeae7f-b0af-4f04-8f6e-ab5fece36311 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.189261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaeae7f-b0af-4f04-8f6e-ab5fece36311 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:29.269443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:29.287535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6fbd9-f1cc-4b02-8f8c-224e1f7688ab None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:29.317156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaeae7f-b0af-4f04-8f6e-ab5fece36311 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:29.318792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaeae7f-b0af-4f04-8f6e-ab5fece36311 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:29.345428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd9e73-f4e3-46f1-a1a4-2e9dd64c81fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:29.346741 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd9e73-f4e3-46f1-a1a4-2e9dd64c81fe None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.347176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd9e73-f4e3-46f1-a1a4-2e9dd64c81fe None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.347577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd9e73-f4e3-46f1-a1a4-2e9dd64c81fe None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:29.349796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd9e73-f4e3-46f1-a1a4-2e9dd64c81fe None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:29.352365 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd9e73-f4e3-46f1-a1a4-2e9dd64c81fe None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:29.355671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 228/441] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:29 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:29.357546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08100-8f0f-4077-b23e-746c85adbfa7 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.358944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08100-8f0f-4077-b23e-746c85adbfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.358944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf08100-8f0f-4077-b23e-746c85adbfa7 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:29.360632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 229/442] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:29.362126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf08100-8f0f-4077-b23e-746c85adbfa7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:29.446642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:29.464367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 214/443] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:29.466252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a841-e7d8-48fa-b65a-6a595bffdd53 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.466703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a841-e7d8-48fa-b65a-6a595bffdd53 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.466979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65a841-e7d8-48fa-b65a-6a595bffdd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:29.489351 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ce65a841-e7d8-48fa-b65a-6a595bffdd53 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:29.490840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ce65a841-e7d8-48fa-b65a-6a595bffdd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:29.525093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf08100-8f0f-4077-b23e-746c85adbfa7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:29.569254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db591f5-f831-4316-916f-4dc6c9182a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:29.570291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db591f5-f831-4316-916f-4dc6c9182a80 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.570698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db591f5-f831-4316-916f-4dc6c9182a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.571053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db591f5-f831-4316-916f-4dc6c9182a80 None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:29.571473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 215/444] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:29.572503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a841-e7d8-48fa-b65a-6a595bffdd53 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:29.572970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db591f5-f831-4316-916f-4dc6c9182a80 None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:29.574633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db591f5-f831-4316-916f-4dc6c9182a80 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:29.576787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 230/445] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:29 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:29.582777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db591f5-f831-4316-916f-4dc6c9182a80 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:29.593060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:29.701171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db591f5-f831-4316-916f-4dc6c9182a80 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:29.711250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65a841-e7d8-48fa-b65a-6a595bffdd53 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:29.725525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:29.746159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:29.746840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.747126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.747403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:29.748231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] RBAC: Adding query filter params (group.id=79b400bd2a62473fb5d1f485d779aefd, role.id=9d6a699c2b1b45629628fd317cc9c15b, scope.project.id=5c8b586142aa493484ff06e0a70778cf) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:29.748548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:29.749632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eae0a-94dd-4cb1-8c3c-e0c6e7c8ab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:29.750043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:29.750532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eae0a-94dd-4cb1-8c3c-e0c6e7c8ab44 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.750811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eae0a-94dd-4cb1-8c3c-e0c6e7c8ab44 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.751082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eae0a-94dd-4cb1-8c3c-e0c6e7c8ab44 None admin] PATH_INFO: `/v3/projects/5c8b586142aa493484ff06e0a70778cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:29.752272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eae0a-94dd-4cb1-8c3c-e0c6e7c8ab44 None admin] RBAC: Authorizing `identity:get_project(project_id=5c8b586142aa493484ff06e0a70778cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:29.753802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eae0a-94dd-4cb1-8c3c-e0c6e7c8ab44 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:29.755567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 216/446] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:29 2022] GET /identity/v3/projects/5c8b586142aa493484ff06e0a70778cf => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:29.756416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eae0a-94dd-4cb1-8c3c-e0c6e7c8ab44 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:29.762340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 231/447] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:29 2022] GET /identity/v3/role_assignments?group.id=79b400bd2a62473fb5d1f485d779aefd&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=9d6a699c2b1b45629628fd317cc9c15b => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:29.763141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:29.891156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:29.925208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eae0a-94dd-4cb1-8c3c-e0c6e7c8ab44 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:29.926492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b04da-8c22-41b1-a4f3-aac0fe122ecc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:29.983211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965507a-bc4a-46b1-9aa7-ba464abb8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:29.986199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965507a-bc4a-46b1-9aa7-ba464abb8464 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.986489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965507a-bc4a-46b1-9aa7-ba464abb8464 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.986853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965507a-bc4a-46b1-9aa7-ba464abb8464 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:29.997530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f820d85-6758-4367-8260-47dd95b62523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:29.998862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820d85-6758-4367-8260-47dd95b62523 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:29.999325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820d85-6758-4367-8260-47dd95b62523 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:29.999758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f820d85-6758-4367-8260-47dd95b62523 None admin] PATH_INFO: `/v3/groups/79b400bd2a62473fb5d1f485d779aefd` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:30.001877 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f820d85-6758-4367-8260-47dd95b62523 None admin] RBAC: Authorizing `identity:get_group(group_id=79b400bd2a62473fb5d1f485d779aefd)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:30.004473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f820d85-6758-4367-8260-47dd95b62523 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:30.006557 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965507a-bc4a-46b1-9aa7-ba464abb8464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:30.007384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 232/448] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:29 2022] GET /identity/v3/groups/79b400bd2a62473fb5d1f485d779aefd => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0965507a-bc4a-46b1-9aa7-ba464abb8464 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:30.015902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:30.016933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:05:30.018189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:30.019401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f820d85-6758-4367-8260-47dd95b62523 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:30.019520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 217/449] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:05:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:30.031133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965507a-bc4a-46b1-9aa7-ba464abb8464 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:30.106781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:30.196771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f820d85-6758-4367-8260-47dd95b62523 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:30.201352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965507a-bc4a-46b1-9aa7-ba464abb8464 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:30.257785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:30.258473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:30.258749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:30.259034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:30.260156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] RBAC: Adding query filter params (group.id=79b400bd2a62473fb5d1f485d779aefd, role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=5c8b586142aa493484ff06e0a70778cf) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:30.260442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:30.263245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:30.263517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:30.264206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:30.264491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:30.264772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:30.265547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:30.265963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:30.267399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:30.271337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 233/450] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:30 2022] GET /identity/v3/role_assignments?group.id=79b400bd2a62473fb5d1f485d779aefd&scope.project.id=5c8b586142aa493484ff06e0a70778cf&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 611 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:30.273471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:30.280926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 218/451] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:30.290912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:30.451125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560e9fb-c9b3-4ef8-844b-ad828094b8dd None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:30.471748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:30.500291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567fed8-1112-438a-a7dc-e511be4d20e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:30.501351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567fed8-1112-438a-a7dc-e511be4d20e7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:30.501847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567fed8-1112-438a-a7dc-e511be4d20e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:30.502193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567fed8-1112-438a-a7dc-e511be4d20e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:30.590955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567fed8-1112-438a-a7dc-e511be4d20e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:30.592487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567fed8-1112-438a-a7dc-e511be4d20e7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:30.741700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 234/452] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:30.750118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:30.886576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b137f8d-73f9-4ca2-9f66-d1a05ded6097 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:30.941721 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8067e9-a7a4-4bf6-b277-6209ffa1a4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:30.943267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8067e9-a7a4-4bf6-b277-6209ffa1a4cc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:30.943767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8067e9-a7a4-4bf6-b277-6209ffa1a4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:30.944261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8067e9-a7a4-4bf6-b277-6209ffa1a4cc None admin] PATH_INFO: `/v3/projects/c1db40e87aaf497fb0528cd53913ada9` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:30.946421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8067e9-a7a4-4bf6-b277-6209ffa1a4cc None admin] RBAC: Authorizing `identity:get_project(project_id=c1db40e87aaf497fb0528cd53913ada9)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:30.949544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8067e9-a7a4-4bf6-b277-6209ffa1a4cc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:30.952947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 219/453] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:05:30 2022] GET /identity/v3/projects/c1db40e87aaf497fb0528cd53913ada9 => generated 344 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:30.961807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567fed8-1112-438a-a7dc-e511be4d20e7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:30.978525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:31.077267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567fed8-1112-438a-a7dc-e511be4d20e7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:31.115768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cf83e-2ea5-4207-95a8-b411c1a2dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:31.116647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cf83e-2ea5-4207-95a8-b411c1a2dd7a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:31.116927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cf83e-2ea5-4207-95a8-b411c1a2dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:31.117197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cf83e-2ea5-4207-95a8-b411c1a2dd7a None admin] PATH_INFO: `/v3/groups/8fd181240db74a819704cdb3469a5cf4` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:31.118851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cf83e-2ea5-4207-95a8-b411c1a2dd7a None admin] RBAC: Authorizing `identity:get_group(group_id=8fd181240db74a819704cdb3469a5cf4)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:31.120590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249cf83e-2ea5-4207-95a8-b411c1a2dd7a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:31.122569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 235/454] 10.222.0.71 () {56 vars in 1180 bytes} [Mon Jul 18 14:05:30 2022] GET /identity/v3/groups/8fd181240db74a819704cdb3469a5cf4 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:31.128851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8067e9-a7a4-4bf6-b277-6209ffa1a4cc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:31.201265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:31.243605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8067e9-a7a4-4bf6-b277-6209ffa1a4cc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:31.281984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9c818-47e8-4925-9cc0-d03bce37edf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:31.282666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9c818-47e8-4925-9cc0-d03bce37edf9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:31.282955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9c818-47e8-4925-9cc0-d03bce37edf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:31.283236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9c818-47e8-4925-9cc0-d03bce37edf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:31.284085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9c818-47e8-4925-9cc0-d03bce37edf9 None admin] RBAC: Adding query filter params (group.id=8fd181240db74a819704cdb3469a5cf4, role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=c1db40e87aaf497fb0528cd53913ada9) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:31.284402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9c818-47e8-4925-9cc0-d03bce37edf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:31.286011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9c818-47e8-4925-9cc0-d03bce37edf9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:31.293068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 220/455] 10.222.0.71 () {56 vars in 1376 bytes} [Mon Jul 18 14:05:31 2022] GET /identity/v3/role_assignments?group.id=8fd181240db74a819704cdb3469a5cf4&scope.project.id=c1db40e87aaf497fb0528cd53913ada9&role.id=5c672a1b2979444383c587a215adeb35 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:34.254131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35421282-69be-4728-bbf5-bc0b677d6ed9 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:34.254671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35421282-69be-4728-bbf5-bc0b677d6ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:34.255022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35421282-69be-4728-bbf5-bc0b677d6ed9 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:34.257723 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 236/456] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:34.266508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed25a-ccd4-45f4-bf2f-b0ca856a58f8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:34.267726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed25a-ccd4-45f4-bf2f-b0ca856a58f8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:34.268332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ed25a-ccd4-45f4-bf2f-b0ca856a58f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:34.304924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bd9ed25a-ccd4-45f4-bf2f-b0ca856a58f8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:34.307441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bd9ed25a-ccd4-45f4-bf2f-b0ca856a58f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:34.413708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 221/457] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:34.463098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35421282-69be-4728-bbf5-bc0b677d6ed9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:34.616841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:34.653694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35421282-69be-4728-bbf5-bc0b677d6ed9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:34.653949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af538c5-3182-4edc-9dd0-38a262ab4c05 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:34.653949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af538c5-3182-4edc-9dd0-38a262ab4c05 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:34.653949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af538c5-3182-4edc-9dd0-38a262ab4c05 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:34.659562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 222/458] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:34.669621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22275f-2d27-40d0-9480-592ab3945f83 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:34.669998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22275f-2d27-40d0-9480-592ab3945f83 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:34.670285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22275f-2d27-40d0-9480-592ab3945f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:34.700301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0b22275f-2d27-40d0-9480-592ab3945f83 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:34.701726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0b22275f-2d27-40d0-9480-592ab3945f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:34.724582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b66e2d-faeb-4122-89b7-b1138af65860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:34.726368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b66e2d-faeb-4122-89b7-b1138af65860 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:34.727167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b66e2d-faeb-4122-89b7-b1138af65860 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:34.728471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b66e2d-faeb-4122-89b7-b1138af65860 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:34.744253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b66e2d-faeb-4122-89b7-b1138af65860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:34.746291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b66e2d-faeb-4122-89b7-b1138af65860 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-58b66e2d-faeb-4122-89b7-b1138af65860 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:34.771750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:34.772680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:34.773521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:34.773521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:34.773521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:34.773521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:34.781286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 237/459] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:34.781286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb788-2c3f-45cf-a973-d4e87c642efe None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:34.781286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb788-2c3f-45cf-a973-d4e87c642efe None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:34.781286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb788-2c3f-45cf-a973-d4e87c642efe None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:34.788119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 238/460] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:34.790003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efb788-2c3f-45cf-a973-d4e87c642efe None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:34.805214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:34.822565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 223/461] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:34.824454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be64a2c-1c78-443b-a36d-f7b47af2b442 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:34.824923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be64a2c-1c78-443b-a36d-f7b47af2b442 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:34.825372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be64a2c-1c78-443b-a36d-f7b47af2b442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:34.852534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4be64a2c-1c78-443b-a36d-f7b47af2b442 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:34.854096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4be64a2c-1c78-443b-a36d-f7b47af2b442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:34.957835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 224/462] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:34.958921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be64a2c-1c78-443b-a36d-f7b47af2b442 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:34.968880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efb788-2c3f-45cf-a973-d4e87c642efe None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:35.012568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:35.018515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:35.019201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.019493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.019772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.020507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:35.020820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:35.022269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:35.047256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 239/463] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:35.048285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:35.101109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be64a2c-1c78-443b-a36d-f7b47af2b442 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:35.127637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:35.145854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:35.147077 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.147571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.148095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.161590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:35.162296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3a7c4-8e6e-4355-9ba5-959dd66b2146 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:35.163717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:35.185458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:35.186426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:35.187244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:35.187244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:35.187244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:35.187244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:35.195121 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 225/464] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:35.196531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:35.204014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:35.204839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.205127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.205405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.217489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:35.218957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:35.238493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:35.239502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:35.240340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:35.240340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:35.240340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:35.240340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:35.245298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 240/465] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:35.246637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:35.300635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:35.318031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f6b12-867b-4bab-af61-ecd47a15f27b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:35.362628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c5970-92b9-4b4d-b150-a150c4d68563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:35.363324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5970-92b9-4b4d-b150-a150c4d68563 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.363607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5970-92b9-4b4d-b150-a150c4d68563 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.363883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5970-92b9-4b4d-b150-a150c4d68563 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.364610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c5970-92b9-4b4d-b150-a150c4d68563 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:35.366148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c5970-92b9-4b4d-b150-a150c4d68563 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:35.375778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da968f-9608-405a-b413-a6713a0e8dfa None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:35.394764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 226/466] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:05:35 2022] POST /identity/v3/users => generated 246 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:05:35.396049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f66bfa-8cc7-422c-a5c9-a233ab908f4e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.396439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f66bfa-8cc7-422c-a5c9-a233ab908f4e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.396756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f66bfa-8cc7-422c-a5c9-a233ab908f4e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.401934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 227/467] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:35.401934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f66bfa-8cc7-422c-a5c9-a233ab908f4e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:35.462922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:35.469745 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7fb6f-2f12-4d11-aa79-6b2c8b3b3ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:35.474098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fb6f-2f12-4d11-aa79-6b2c8b3b3ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.475605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fb6f-2f12-4d11-aa79-6b2c8b3b3ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.476315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7fb6f-2f12-4d11-aa79-6b2c8b3b3ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.478463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7fb6f-2f12-4d11-aa79-6b2c8b3b3ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:35.480181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7fb6f-2f12-4d11-aa79-6b2c8b3b3ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:35.484255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7fb6f-2f12-4d11-aa79-6b2c8b3b3ba4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:35.512200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 241/468] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:35.515109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b757d6-abda-4b2e-8b7c-2403aba92373 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.515875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b757d6-abda-4b2e-8b7c-2403aba92373 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.516516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b757d6-abda-4b2e-8b7c-2403aba92373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.547796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-57b757d6-abda-4b2e-8b7c-2403aba92373 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:35.550375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-57b757d6-abda-4b2e-8b7c-2403aba92373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:35.595407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f66bfa-8cc7-422c-a5c9-a233ab908f4e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:35.655203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:35.656280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.656768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.657239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.658258 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:35.658772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:35.660473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:35.667364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 242/469] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:35.670978 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b757d6-abda-4b2e-8b7c-2403aba92373 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:35.675948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 228/470] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:35.681508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:35.753514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:35.820787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b757d6-abda-4b2e-8b7c-2403aba92373 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:35.841204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:35.859564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffb727-d623-47ff-8eaf-4eceffdf6191 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:35.867185 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac409037-fc15-47a4-8be2-43542146aaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:35.869250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac409037-fc15-47a4-8be2-43542146aaa7 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.869720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac409037-fc15-47a4-8be2-43542146aaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.870168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac409037-fc15-47a4-8be2-43542146aaa7 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.871619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac409037-fc15-47a4-8be2-43542146aaa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:35.873949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac409037-fc15-47a4-8be2-43542146aaa7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:35.903170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 243/471] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:05:35 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:05:35.904844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db871c81-7b25-46bd-89ea-137c2dab99b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:35.905559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871c81-7b25-46bd-89ea-137c2dab99b5 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.905895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871c81-7b25-46bd-89ea-137c2dab99b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.906179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db871c81-7b25-46bd-89ea-137c2dab99b5 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.906932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db871c81-7b25-46bd-89ea-137c2dab99b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:35.908637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db871c81-7b25-46bd-89ea-137c2dab99b5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:35.914850 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac409037-fc15-47a4-8be2-43542146aaa7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:35.937445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 229/472] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:05:35 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:05:35.947423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688068-3a1c-4e6d-a3e5-5eaed183d86b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.947729 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688068-3a1c-4e6d-a3e5-5eaed183d86b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.948024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c688068-3a1c-4e6d-a3e5-5eaed183d86b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:35.955920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 230/473] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:35 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:35.958227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99c1a-295b-4bb5-94dd-e467ae1e2722 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:35.958579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99c1a-295b-4bb5-94dd-e467ae1e2722 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:35.958846 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df99c1a-295b-4bb5-94dd-e467ae1e2722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:36.018556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9df99c1a-295b-4bb5-94dd-e467ae1e2722 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:36.019790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9df99c1a-295b-4bb5-94dd-e467ae1e2722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:36.156495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:36.186352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 231/474] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:36.191380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac409037-fc15-47a4-8be2-43542146aaa7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:36.224926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df99c1a-295b-4bb5-94dd-e467ae1e2722 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:36.236523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:36.238144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:36.238761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:36.239853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:36.262494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:36.265310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:36.297774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:36.298750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:36.299583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:36.299583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:36.299583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:36.299583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:36.301443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 244/475] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:36.308355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:36.382999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:36.434143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df99c1a-295b-4bb5-94dd-e467ae1e2722 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:36.491804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f15e6-3fad-46d5-8556-a85155c292e6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:36.521335 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:36.522733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:36.523021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:36.523302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:36.524097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:36.526042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:36.542050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.sql.core [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 14:05:36.542050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 14:05:36.542050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [parameters: {'id': '5ad61d22319b4c93ab0773e2853d5cc4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 14:05:36.542050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80213) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 14:05:36.571942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:36.572804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:36.573148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:36.573433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:36.574225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:36.574590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:36.576048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result.read() Jul 18 14:05:36.578848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return ref.to_dict() Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:05:36.579706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application yield resource Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise self.value Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application session.commit() Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 14:05:36.581225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.session.flush() Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application compat.raise_( Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 14:05:36.583499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:05:36.590575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result.read() Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:05:36.593287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [parameters: {'id': '5ad61d22319b4c93ab0773e2853d5cc4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:36.597378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 18 14:05:36.599424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:36.601410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 232/476] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 18 14:05:36 2022] POST /identity/v3/roles => generated 144 bytes in 356 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:36.601410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:36.612222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 245/477] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:36.617619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:36.623721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:36.813042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf64a3-fc76-4614-a673-3273108638b2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:36.840791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08603885-55d4-4ec7-8aa6-48d837ee0b34 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:36.886338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e96bf5-4bcb-4f0c-8583-e994db450c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:36.888143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e96bf5-4bcb-4f0c-8583-e994db450c8e None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:36.888624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e96bf5-4bcb-4f0c-8583-e994db450c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:36.889113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e96bf5-4bcb-4f0c-8583-e994db450c8e None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:36.893347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e96bf5-4bcb-4f0c-8583-e994db450c8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:36.896117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e96bf5-4bcb-4f0c-8583-e994db450c8e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:36.954749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 246/478] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:05:36 2022] POST /identity/v3/users => generated 249 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:05:36.959553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1696b4-38d4-4699-bef7-0be064c4035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:36.960841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1696b4-38d4-4699-bef7-0be064c4035f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:36.961185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1696b4-38d4-4699-bef7-0be064c4035f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:36.961632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1696b4-38d4-4699-bef7-0be064c4035f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:36.981418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1696b4-38d4-4699-bef7-0be064c4035f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:36.990147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3d1696b4-38d4-4699-bef7-0be064c4035f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:37.004076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:37.005791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:37.008168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 233/479] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:05:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:37.015335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e96bf5-4bcb-4f0c-8583-e994db450c8e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:37.149097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:37.224853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e96bf5-4bcb-4f0c-8583-e994db450c8e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:37.266285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655a93a-4aba-4456-a553-cb4684f0721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:37.266940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655a93a-4aba-4456-a553-cb4684f0721e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:37.267224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655a93a-4aba-4456-a553-cb4684f0721e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:37.267492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655a93a-4aba-4456-a553-cb4684f0721e None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:37.268258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655a93a-4aba-4456-a553-cb4684f0721e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:37.268560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655a93a-4aba-4456-a553-cb4684f0721e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:37.270030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655a93a-4aba-4456-a553-cb4684f0721e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:37.280800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 247/480] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:38.496008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32dee-9ea4-402a-8cd0-553be7c3af22 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:38.496624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32dee-9ea4-402a-8cd0-553be7c3af22 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:38.496624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b32dee-9ea4-402a-8cd0-553be7c3af22 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:38.499483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 234/481] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:38.508918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cec8d-279d-4fcf-98c7-52c4e631d087 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:38.510219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cec8d-279d-4fcf-98c7-52c4e631d087 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:38.510740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262cec8d-279d-4fcf-98c7-52c4e631d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:38.555254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-262cec8d-279d-4fcf-98c7-52c4e631d087 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:38.557595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-262cec8d-279d-4fcf-98c7-52c4e631d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:38.677476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 248/482] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:38.718240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b32dee-9ea4-402a-8cd0-553be7c3af22 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:38.910926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602b58b-7f3a-4fbc-99ab-3c629c9122ec None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:38.911340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602b58b-7f3a-4fbc-99ab-3c629c9122ec None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:38.911517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2602b58b-7f3a-4fbc-99ab-3c629c9122ec None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:38.913097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 249/483] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:05:38.916392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b32dee-9ea4-402a-8cd0-553be7c3af22 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:38.919305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4740-91f1-470d-beff-d323099142b1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:38.919912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4740-91f1-470d-beff-d323099142b1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:38.920393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440a4740-91f1-470d-beff-d323099142b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:38.950954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-440a4740-91f1-470d-beff-d323099142b1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:38.953400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-440a4740-91f1-470d-beff-d323099142b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:38.968408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a6874-3a28-452a-bbdc-d5a36be52d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:38.969385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6874-3a28-452a-bbdc-d5a36be52d0b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:38.969941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6874-3a28-452a-bbdc-d5a36be52d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:38.970232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a6874-3a28-452a-bbdc-d5a36be52d0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:38.982974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a6874-3a28-452a-bbdc-d5a36be52d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4b5a6874-3a28-452a-bbdc-d5a36be52d0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:38.992204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:38.993220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:38.995234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:39.001764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 235/484] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:39.005459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58304856-477d-47a9-94ab-ce410777c3dd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:39.006734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58304856-477d-47a9-94ab-ce410777c3dd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:39.007688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58304856-477d-47a9-94ab-ce410777c3dd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:39.015221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 236/485] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:39.015221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58304856-477d-47a9-94ab-ce410777c3dd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:39.052167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:39.090867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 250/486] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:39.093117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc9c37-c719-458c-b2e0-7d25890da652 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:39.093599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc9c37-c719-458c-b2e0-7d25890da652 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:39.094203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc9c37-c719-458c-b2e0-7d25890da652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:39.123570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:39.125045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b9dc9c37-c719-458c-b2e0-7d25890da652 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:39.127682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b9dc9c37-c719-458c-b2e0-7d25890da652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:39.181092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58304856-477d-47a9-94ab-ce410777c3dd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:39.225258 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:39.226660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:39.227288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:39.229010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:39.237572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:39.238428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:39.239237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 251/487] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:39.240256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc9c37-c719-458c-b2e0-7d25890da652 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:39.241747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:39.255421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 237/488] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:39.265502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:39.370061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc9c37-c719-458c-b2e0-7d25890da652 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:39.390699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:39.431638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa48c9-fb0e-4e4d-b18a-1afa3bdc2465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:39.431638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa48c9-fb0e-4e4d-b18a-1afa3bdc2465 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:39.431638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa48c9-fb0e-4e4d-b18a-1afa3bdc2465 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:39.432165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa48c9-fb0e-4e4d-b18a-1afa3bdc2465 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:39.448956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa48c9-fb0e-4e4d-b18a-1afa3bdc2465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:39.464016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfaa03b-f004-4820-8486-29b223d5474c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d6aa48c9-fb0e-4e4d-b18a-1afa3bdc2465 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:39.466632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:39.468295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:39.469820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 252/489] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:39.472869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa48c9-fb0e-4e4d-b18a-1afa3bdc2465 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:39.501435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:39.528726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:39.530119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:39.530584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:39.530872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:39.548123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:39.551920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:39.582703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:39.584288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:39.586050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:39.586050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:39.586050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:39.586050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:39.586050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 238/490] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:39.587447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:39.675612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa48c9-fb0e-4e4d-b18a-1afa3bdc2465 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:39.741996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e6855-cb31-475b-9eee-96c15b58808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:39.743224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6855-cb31-475b-9eee-96c15b58808e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:39.744836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6855-cb31-475b-9eee-96c15b58808e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:39.745451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e6855-cb31-475b-9eee-96c15b58808e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:39.760957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e6855-cb31-475b-9eee-96c15b58808e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:39.770550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d1afd-0365-429c-b6b9-9aa06f2d55c8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-873e6855-cb31-475b-9eee-96c15b58808e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:39.777317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:39.779087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:39.780633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:39.785695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 253/491] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:39.787665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e6855-cb31-475b-9eee-96c15b58808e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:39.816984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:39.823635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:39.824095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:39.825188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:39.828668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:39.829137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:39.830806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:39.843715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 239/492] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:39.844981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:39.863458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:39.920315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:39.988131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e6855-cb31-475b-9eee-96c15b58808e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.031665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.032417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.032823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.033214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.034090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:40.034505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.036047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.059170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 254/493] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:40.059170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:40.064762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b0367-d069-43d9-aad9-111e8ef00b8d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.106874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.107932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.108324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.108842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.109584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:40.110032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.111653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.122905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 240/494] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:40.124007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:40.219866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:40.231654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55403555-e534-45cb-a4e9-66e52a513256 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.254356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:40.262298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e324f-59cf-41f1-a2c0-1ed4e6be47a7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.271539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.272375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.272651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.272919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.284929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.286392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.311284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60026183-684c-4dc4-b415-3b698606be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.312542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60026183-684c-4dc4-b415-3b698606be16 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.312999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60026183-684c-4dc4-b415-3b698606be16 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.313446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60026183-684c-4dc4-b415-3b698606be16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:40.315710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:40.316677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:40.317536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:40.317536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:40.317536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:40.317536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:40.317536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 255/495] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:40.319184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:40.333333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60026183-684c-4dc4-b415-3b698606be16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.334941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60026183-684c-4dc4-b415-3b698606be16 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-60026183-684c-4dc4-b415-3b698606be16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:40.354712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:40.355751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:40.356747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 241/496] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:40.357744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700b8cf-5321-403e-9d6b-41391bbe00b1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.357840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700b8cf-5321-403e-9d6b-41391bbe00b1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.358141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700b8cf-5321-403e-9d6b-41391bbe00b1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.359670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 242/497] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:40.360913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa107f-1cba-4ab6-8344-afce1ac1b176 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.361195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa107f-1cba-4ab6-8344-afce1ac1b176 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.361465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa107f-1cba-4ab6-8344-afce1ac1b176 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.362875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 243/498] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:40.363648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa107f-1cba-4ab6-8344-afce1ac1b176 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:40.461896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7080583e-099a-42e7-bfb6-9e08c55397e8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.492391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa107f-1cba-4ab6-8344-afce1ac1b176 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.500789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.501600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.501907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.502178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.514533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.516028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.518188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:40.532552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc895e28-4155-4ca1-b523-f178300a9854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.533234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc895e28-4155-4ca1-b523-f178300a9854 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.533521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc895e28-4155-4ca1-b523-f178300a9854 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.533870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc895e28-4155-4ca1-b523-f178300a9854 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.534626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc895e28-4155-4ca1-b523-f178300a9854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:40.535136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc895e28-4155-4ca1-b523-f178300a9854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.536513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc895e28-4155-4ca1-b523-f178300a9854 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:40.537584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:40.538544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:40.539456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:40.539456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:40.539456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:40.539456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:40.544121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 256/499] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:40.544912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:40.550242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 244/500] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:40.551564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a9b9-e7f9-410b-b69c-2a0eff4508cc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.551860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a9b9-e7f9-410b-b69c-2a0eff4508cc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.552269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591a9b9-e7f9-410b-b69c-2a0eff4508cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.570080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0591a9b9-e7f9-410b-b69c-2a0eff4508cc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:40.570697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0591a9b9-e7f9-410b-b69c-2a0eff4508cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:40.583576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:40.655290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 245/501] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:40.656843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c28659-8fd8-478a-a0f4-9cd26f93aa29 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.657134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c28659-8fd8-478a-a0f4-9cd26f93aa29 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.657412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c28659-8fd8-478a-a0f4-9cd26f93aa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.684077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd8801-4d8b-4db6-b3dc-2d3e044744e4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.684629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d9c28659-8fd8-478a-a0f4-9cd26f93aa29 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:40.685461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d9c28659-8fd8-478a-a0f4-9cd26f93aa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:40.729964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.730646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.730933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.731219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.732603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:40.732912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.734432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.763092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 246/502] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:40.764520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c28659-8fd8-478a-a0f4-9cd26f93aa29 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:40.776585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 257/503] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:40.777292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:40.836032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:40.889796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811eaa5-d030-4418-a568-aa004f51fdd5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.890496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:40.918564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c28659-8fd8-478a-a0f4-9cd26f93aa29 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:40.929830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.930820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.931244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.931614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.939580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.941072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.955333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:40.956006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:40.956006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:40.956006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:40.957259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:40.957571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:40.957799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:40.960815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:40.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 258/504] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:40.963950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:40.964082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:40.976428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 247/505] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:40.977305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.088362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a58503-305f-40c2-b1e3-aa9aaac80b9e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.101261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c0975-026d-4ddf-b9fa-2a6b672abab7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.102582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:41.126822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.127627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.127910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.128179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.140451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911372c4-deb6-4dc2-a302-2139d89e2887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.141120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911372c4-deb6-4dc2-a302-2139d89e2887 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.141404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911372c4-deb6-4dc2-a302-2139d89e2887 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.141759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911372c4-deb6-4dc2-a302-2139d89e2887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.145636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:41.146251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:41.147757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:41.167222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:41.168171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:41.169023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:41.169023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:41.169023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:41.169023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:41.169023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 259/506] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:41.169318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.239733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911372c4-deb6-4dc2-a302-2139d89e2887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:41.241341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911372c4-deb6-4dc2-a302-2139d89e2887 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:41.300168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071e64d-af52-4b60-b957-fdcdf5c93ba4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.350059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb129bb-421a-451c-a457-a8947b01b72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.351222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb129bb-421a-451c-a457-a8947b01b72e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.351585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb129bb-421a-451c-a457-a8947b01b72e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.351868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb129bb-421a-451c-a457-a8947b01b72e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.367321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb129bb-421a-451c-a457-a8947b01b72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:41.373963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 248/507] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:41.374797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911372c4-deb6-4dc2-a302-2139d89e2887 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-bcb129bb-421a-451c-a457-a8947b01b72e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:41.375655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:41.376557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:41.377351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 260/508] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:41.378051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb129bb-421a-451c-a457-a8947b01b72e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.405913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:41.425134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:41.495096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb129bb-421a-451c-a457-a8947b01b72e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.503010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911372c4-deb6-4dc2-a302-2139d89e2887 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.532854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.533668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.533975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.534250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.542469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.543174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.543462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.543737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.544599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:41.544723 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:41.544916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:41.546427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:41.546563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:41.563054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:41.570768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 261/509] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:41.572474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.586880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 249/510] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 18 14:05:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:41.587827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.694198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:41.708397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:41.711969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ab63b-9e62-4a79-9b66-9c031a603921 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.715598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e886f5-c78c-4f4a-b5ce-a67174fbf193 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.746787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.747482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.747768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.748019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.748769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:41.749080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:41.750544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:41.752495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.753686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.753979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.754252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.762866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 262/511] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:41.763719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.769547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:41.771161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:41.789544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:41.790489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:41.791801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:41.791801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:41.791801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:41.791801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:41.791801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 250/512] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:41.791801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.880159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096e69a-7186-4479-bff7-7f89fef86b0c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.896786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41e2e2-29f0-4af1-9308-182020bdded3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:41.918572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.919707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.920200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.920688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.922144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:41.922698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:41.925286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:41.933917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:41.934601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:41.934889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:41.935172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:41.936016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:41.936338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:41.937794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:41.944401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 263/513] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:41.944401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.954697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:41.962376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 251/514] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 18 14:05:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:41.963361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:41.968353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:42.059172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c25c51-8f8d-4bcd-8065-e2c7f4387c13 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:42.072356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d580db-65d8-48e6-b6d4-156b9e9324f2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:42.098901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6afcd-db23-4092-90ed-83ef95974bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:42.099559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6afcd-db23-4092-90ed-83ef95974bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:42.099843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6afcd-db23-4092-90ed-83ef95974bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:42.100120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6afcd-db23-4092-90ed-83ef95974bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:42.110189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f2c8e-25fe-44eb-8b04-9699972fbd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:42.110870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f2c8e-25fe-44eb-8b04-9699972fbd28 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:42.111148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f2c8e-25fe-44eb-8b04-9699972fbd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:42.111418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f2c8e-25fe-44eb-8b04-9699972fbd28 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:42.112125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f2c8e-25fe-44eb-8b04-9699972fbd28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:42.113564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f2c8e-25fe-44eb-8b04-9699972fbd28 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:42.134934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 252/515] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:05:41 2022] POST /identity/v3/users => generated 247 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:05:42.135791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f2c8e-25fe-44eb-8b04-9699972fbd28 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:42.191215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6afcd-db23-4092-90ed-83ef95974bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:42.193444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6afcd-db23-4092-90ed-83ef95974bf7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:42.203508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:42.210159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:42.279486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f2c8e-25fe-44eb-8b04-9699972fbd28 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:42.332627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 264/516] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:42.333483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6afcd-db23-4092-90ed-83ef95974bf7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:42.341789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:42.341789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:42.341789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:42.341789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:42.342186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:42.342459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:42.343883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:42.362270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 253/517] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:42.363315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:42.457431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6afcd-db23-4092-90ed-83ef95974bf7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:42.468279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:42.483317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe06f62-dc2c-4b5e-a705-8b216d2dddd7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:42.502063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:42.502878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:42.503162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:42.503436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:42.509428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:42.515197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:42.516698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:42.526838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e43c4-5cec-4bd6-922e-f6f48dd5001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:42.527541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e43c4-5cec-4bd6-922e-f6f48dd5001e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:42.527789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e43c4-5cec-4bd6-922e-f6f48dd5001e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:42.528063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e43c4-5cec-4bd6-922e-f6f48dd5001e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:42.535439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:42.536330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:42.537205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:42.537205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:42.537205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:42.537205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:42.537205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 265/518] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:42.537464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:42.608883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e43c4-5cec-4bd6-922e-f6f48dd5001e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:42.611056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e43c4-5cec-4bd6-922e-f6f48dd5001e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:42.646586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82ea1f-bdb3-4700-a51a-2cd580590710 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:42.685903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:42.686816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:42.687105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:42.687379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:42.699018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:42.700530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:42.719855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:42.720813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:42.727032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:42.727032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:42.727032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:42.727032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:42.727032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 266/519] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:42.727032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:42.740194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:42.744948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 254/520] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:42.746040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e43c4-5cec-4bd6-922e-f6f48dd5001e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:42.775363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:42.856084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaeb7fc-b0e7-4284-841f-f6e65760cda8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:42.875249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e43c4-5cec-4bd6-922e-f6f48dd5001e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:42.894770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:42.895597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:42.895872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:42.896141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:42.905398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:42.907481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:42.915545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:42.916480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:42.916480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:42.916756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:42.917496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:42.917838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:42.919267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:42.927436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:42.928373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:42.929214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:42.929214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:42.929214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:42.929214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:42.929214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:42.929214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:42.929214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:42.929214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:42.936666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 255/521] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:42.936666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:42.936908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 267/522] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:42.936908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.032983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:43.047010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:43.071871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395786b-f889-424c-bff5-9ba802c14083 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:43.089685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ed919-351d-43f3-90dc-3d7236798c50 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:43.123219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:43.123908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:43.124192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:43.124467 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:43.125440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:43.125793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:43.127206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:43.141693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 256/523] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:43.142940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.150808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:43.151876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:43.151876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:43.152151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:43.169540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:43.171833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:43.197974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:43.199325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:43.205065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:43.205065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:43.205065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:43.205065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:43.205065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 268/524] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:43.205065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.288429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9803d8-c723-491d-b2c5-d8be95743d63 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:43.340138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:43.340830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:43.341113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:43.341388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:43.342327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:43.342644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:43.344185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:43.359640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0139fff-6eb3-4887-af91-3fd73e70f4b2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:43.362161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:43.372298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 257/525] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:43.373105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.401099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:43.418277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:43.419107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:43.419422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:43.419794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:43.430995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:43.432834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:43.449136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:43.450292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 269/526] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:43.451201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.525087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014d6b0-c8d2-4d61-8212-a17fbbf9633d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:43.570574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:43.571627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:43.571931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb61de-b21c-43e5-ad26-3d1b1f3fa69f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:43.572379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:43.572794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:43.581921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:43.584321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:43.605398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:43.606428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:43.607273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:43.607273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:43.607273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:43.607273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:43.607273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:43.607273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:43.607273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:43.607273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:43.608167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:43.608834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:43.609116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:43.609390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:43.609800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 258/527] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:43.610286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:43.610563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:43.612404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:43.612913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.642427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 270/528] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:43.642778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.690820 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:43.692260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:43.738959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2808bfe-ab96-4690-b99b-080f1b1e2682 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:43.758034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df36909-4366-4508-8637-d8a24c8cc9e7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:43.776776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:43.777454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:43.777765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:43.778039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:43.778881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=4fffb91636744fc39886bdf568317ab5) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:43.779200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:43.780663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:43.787014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 259/529] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:43 2022] GET /identity/v3/role_assignments?user.id=4fffb91636744fc39886bdf568317ab5&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:43.787681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.816988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:43.818082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:43.818699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:43.819127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:43.820246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:43.820833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:43.822816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:43.850103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 271/530] 10.222.0.71 () {56 vars in 1139 bytes} [Mon Jul 18 14:05:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:43.851348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:43.977389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3977838-20df-41a0-a5a7-55d0b71f9b26 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:44.006579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:44.012675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b46c6-9e79-4108-84da-33e665a5947d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:44.020340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:44.051339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:44.052242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.052663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.053008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.054020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:44.054404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:44.056374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:44.072883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:44.074035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.074525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.074937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.076900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 260/531] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:44.078066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:44.085644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:44.088116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:44.108231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:44.109313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:44.110416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:44.110416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:44.110416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:44.110416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:44.110416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:44.110416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:44.110416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:44.110416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:44.117874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 272/532] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:44.119262 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:44.204211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967b1525-326c-45b6-a6c6-33638eaeefe3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:44.244293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d87109-9e10-484c-ac28-2cf7771e2818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:44.245019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d87109-9e10-484c-ac28-2cf7771e2818 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.245306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d87109-9e10-484c-ac28-2cf7771e2818 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.245585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d87109-9e10-484c-ac28-2cf7771e2818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.265768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24801842-bcc5-43b0-90ae-2b608f939b8e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:44.303523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:44.312703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:44.313519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.313857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.314115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.315549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=812ba8119a5a4297b409525861ec8118) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:44.315864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:44.317339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:44.320946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d87109-9e10-484c-ac28-2cf7771e2818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:44.327631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d87109-9e10-484c-ac28-2cf7771e2818 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:44.337992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 273/533] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:44 2022] GET /identity/v3/role_assignments?user.id=812ba8119a5a4297b409525861ec8118&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:44.337992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:44.397932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:44.455936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 261/534] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:44.457515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaf80a-9386-44bc-bb8c-14bd91aecd44 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.457834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaf80a-9386-44bc-bb8c-14bd91aecd44 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.458113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eaf80a-9386-44bc-bb8c-14bd91aecd44 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.460121 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 262/535] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:44.461009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaf80a-9386-44bc-bb8c-14bd91aecd44 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:44.461550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7db469-decd-4796-92a1-f912735a828e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:44.508527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46553835-6f8f-4109-9d9a-ce6f6b616ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:44.509499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46553835-6f8f-4109-9d9a-ce6f6b616ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.509833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46553835-6f8f-4109-9d9a-ce6f6b616ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.509914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46553835-6f8f-4109-9d9a-ce6f6b616ad0 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.510882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46553835-6f8f-4109-9d9a-ce6f6b616ad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:44.511190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46553835-6f8f-4109-9d9a-ce6f6b616ad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:44.512668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46553835-6f8f-4109-9d9a-ce6f6b616ad0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:44.535157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 274/536] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:44.536534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea034c3-d738-4970-ae69-47b29683defe None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.536828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea034c3-d738-4970-ae69-47b29683defe None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.537111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea034c3-d738-4970-ae69-47b29683defe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.556788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bea034c3-d738-4970-ae69-47b29683defe None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:44.558236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bea034c3-d738-4970-ae69-47b29683defe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:44.590182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:44.606848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eaf80a-9386-44bc-bb8c-14bd91aecd44 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:44.650502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 275/537] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:44.651456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea034c3-d738-4970-ae69-47b29683defe None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:44.654328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:44.655248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.655630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.656014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.668769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:44.670777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:44.688935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:44.702394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:44.703583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:44.704511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:44.704511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:44.704511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:44.704511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:44.707118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 263/538] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:44.708332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:44.787501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea034c3-d738-4970-ae69-47b29683defe None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:44.824508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:44.825165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.825534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.825751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.826577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=d661615b2dd94d9db6183f580ab0fee6) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:44.826889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:44.828815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:44.832023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effba198-2905-42e4-9b3f-82673f057ba5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:44.842073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 276/539] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:44 2022] GET /identity/v3/role_assignments?user.id=d661615b2dd94d9db6183f580ab0fee6&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:44.843402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:44.875550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bc74b-a7e6-4232-ab11-45a7bed2876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:44.876874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc74b-a7e6-4232-ab11-45a7bed2876e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:44.877412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc74b-a7e6-4232-ab11-45a7bed2876e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:44.877953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bc74b-a7e6-4232-ab11-45a7bed2876e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:44.885822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:44.914447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bc74b-a7e6-4232-ab11-45a7bed2876e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b32bc74b-a7e6-4232-ab11-45a7bed2876e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:44.923260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:44.924242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:44.925016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:44.933925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 264/540] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:44.938829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bc74b-a7e6-4232-ab11-45a7bed2876e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:44.998082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:45.068114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8dc36-0d14-49b6-a5e7-02fc6b1edc30 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:45.148282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bc74b-a7e6-4232-ab11-45a7bed2876e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:45.150109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:45.150811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:45.151542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:45.151951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:45.154096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:45.154451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:45.156003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:45.175501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 277/541] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:45.187176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:45.234322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:45.235016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:45.235298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:45.235572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:45.236343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:45.237908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:45.239343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:45.259492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 265/542] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:45.265887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:45.369434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:45.386519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace88ef-8fff-4a31-b507-56289805c3e6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:45.451551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:45.452283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:45.452566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:45.452842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:45.464015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e8b35-7daa-4249-844e-f3448daab109 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:45.469818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:45.471367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:45.486644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:45.498728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:45.499648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:45.500485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 278/543] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:45.506001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:45.562484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:45.563306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:45.563590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:45.563867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:45.590152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:45.592128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:45.620880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:45.621872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:45.622717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:45.622717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:45.622717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:45.622717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:45.628671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 266/544] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:45.651343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:45.729676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a5b1b-3fd7-48a3-bbbe-b1fffb68e099 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:45.814287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:45.815087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:45.815276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:45.815512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:45.816302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:45.816620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:45.818105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:45.831633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 279/545] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:45.837918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:45.918739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:45.946248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb7dd-1595-4522-ac0e-33152c53de6d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:45.982944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:45.983628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:45.983913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:45.984192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:45.985139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=1f1fe084c3164e50bf5d904350925f6a) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:45.985493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:45.987004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:45.994100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 280/546] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:45 2022] GET /identity/v3/role_assignments?user.id=1f1fe084c3164e50bf5d904350925f6a&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:45.994976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f05efc-c923-4dbe-984f-92de383526a7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:46.042230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:46.083712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:46.084711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:46.085140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:46.085563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:46.086823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:46.087369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:46.089766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:46.116685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 267/547] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:46.127548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:46.275904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca596976-3f1c-4cf9-ac52-34fd53d079cc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:46.302518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:46.315895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7a649-0676-4a3a-8c81-7b18f1c41e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:46.317017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7a649-0676-4a3a-8c81-7b18f1c41e19 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:46.317385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7a649-0676-4a3a-8c81-7b18f1c41e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:46.317792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7a649-0676-4a3a-8c81-7b18f1c41e19 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:46.326807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7a649-0676-4a3a-8c81-7b18f1c41e19 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:46.328367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7a649-0676-4a3a-8c81-7b18f1c41e19 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c4d7a649-0676-4a3a-8c81-7b18f1c41e19 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:46.354511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:46.355973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:05:46.357054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:46.357563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 281/548] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:05:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:46.362834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:46.555799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed50c0-daa4-4916-82aa-79f2103a9a06 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:46.556063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed50c0-daa4-4916-82aa-79f2103a9a06 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:46.556458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed50c0-daa4-4916-82aa-79f2103a9a06 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:46.558159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 282/549] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:05:46.564811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a7d4-6250-4e70-a655-dc3134247c34 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:46.565408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a7d4-6250-4e70-a655-dc3134247c34 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:46.565408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720a7d4-6250-4e70-a655-dc3134247c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:46.583403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac1965-426f-41d9-ac8c-566c268dbab5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:46.590852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a720a7d4-6250-4e70-a655-dc3134247c34 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:46.592157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a720a7d4-6250-4e70-a655-dc3134247c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:46.676955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:46.677674 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:46.677962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:46.678238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:46.679096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:46.679413 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:46.680905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:46.715979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 268/550] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 18 14:05:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:46.721208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:46.736687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 283/551] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:05:46.764687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720a7d4-6250-4e70-a655-dc3134247c34 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:46.776395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:46.911056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720a7d4-6250-4e70-a655-dc3134247c34 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:46.927308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd3a65-1203-4039-8ead-5db0432f006f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:46.944520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:46.995574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378cf4f-a57b-4db9-8a57-0d8092f9ee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:46.997387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378cf4f-a57b-4db9-8a57-0d8092f9ee57 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:46.998257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378cf4f-a57b-4db9-8a57-0d8092f9ee57 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:46.998535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378cf4f-a57b-4db9-8a57-0d8092f9ee57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.004505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd549b9a-7448-4e00-8ac3-6a9ee215a5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:47.005234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd549b9a-7448-4e00-8ac3-6a9ee215a5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.005483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd549b9a-7448-4e00-8ac3-6a9ee215a5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.005780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd549b9a-7448-4e00-8ac3-6a9ee215a5d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.017061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378cf4f-a57b-4db9-8a57-0d8092f9ee57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2378cf4f-a57b-4db9-8a57-0d8092f9ee57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:47.026246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:47.027542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:47.028560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 284/552] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:47.029705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de417-38a1-4186-a34d-8e9c691eb0b2 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.034851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de417-38a1-4186-a34d-8e9c691eb0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.035133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84de417-38a1-4186-a34d-8e9c691eb0b2 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.040895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 285/553] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:47.040895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84de417-38a1-4186-a34d-8e9c691eb0b2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:47.102837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd549b9a-7448-4e00-8ac3-6a9ee215a5d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:47.104491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd549b9a-7448-4e00-8ac3-6a9ee215a5d6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:47.135708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:47.165486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84de417-38a1-4186-a34d-8e9c691eb0b2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:47.205088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:47.206305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.206802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.207326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.208294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:47.208816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:47.210759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:47.217827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 269/554] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 18 14:05:47.219177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47a292-db9f-4454-a20b-a8363390813e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.219472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47a292-db9f-4454-a20b-a8363390813e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.219755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47a292-db9f-4454-a20b-a8363390813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.221912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 286/555] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:47.226459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:47.237707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5a47a292-db9f-4454-a20b-a8363390813e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:47.238672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5a47a292-db9f-4454-a20b-a8363390813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:47.267111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:47.321702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 270/556] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:47.321702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47a292-db9f-4454-a20b-a8363390813e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:47.361523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f894a4c-b939-45b2-a089-c3048e99c805 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:47.391769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:47.408933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:47.410212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.410764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.411335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.423951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:47.425887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:47.435828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47a292-db9f-4454-a20b-a8363390813e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:47.447106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:47.448296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:47.449427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:47.449427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:47.449427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:47.449427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:47.449427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 287/557] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:47.450358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:47.478147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bd6dc-2102-4f74-b206-2fc2cd439384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:47.479318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bd6dc-2102-4f74-b206-2fc2cd439384 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.479438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bd6dc-2102-4f74-b206-2fc2cd439384 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.479624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bd6dc-2102-4f74-b206-2fc2cd439384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.490324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bd6dc-2102-4f74-b206-2fc2cd439384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:47.491808 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bd6dc-2102-4f74-b206-2fc2cd439384 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3c5bd6dc-2102-4f74-b206-2fc2cd439384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:47.511676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:47.513211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:47.514728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:47.514728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:47.514728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:47.514728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:47.514728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 271/558] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:47.514728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757182a-2481-42ef-98d7-9e226a324398 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.514728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757182a-2481-42ef-98d7-9e226a324398 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.514728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757182a-2481-42ef-98d7-9e226a324398 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.516154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 272/559] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:47.516675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757182a-2481-42ef-98d7-9e226a324398 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:47.549586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:47.569770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e9987-9e31-4875-a076-b25e6147a1b2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:47.609486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b32651-78f6-44fa-9bc7-2c87da3057dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:47.610344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b32651-78f6-44fa-9bc7-2c87da3057dd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.610855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b32651-78f6-44fa-9bc7-2c87da3057dd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.610855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b32651-78f6-44fa-9bc7-2c87da3057dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.623008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b32651-78f6-44fa-9bc7-2c87da3057dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-b9b32651-78f6-44fa-9bc7-2c87da3057dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:47.631351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:47.632348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 288/560] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:47.633279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b32651-78f6-44fa-9bc7-2c87da3057dd None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:47.646267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757182a-2481-42ef-98d7-9e226a324398 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:47.659586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:47.692365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c0701-9fa7-4819-9d84-f17d8421d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:47.692984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0701-9fa7-4819-9d84-f17d8421d3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.693269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0701-9fa7-4819-9d84-f17d8421d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.693542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0701-9fa7-4819-9d84-f17d8421d3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.694316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c0701-9fa7-4819-9d84-f17d8421d3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:47.694804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c0701-9fa7-4819-9d84-f17d8421d3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:47.696245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c0701-9fa7-4819-9d84-f17d8421d3f5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:47.711171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 273/561] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:47.712877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44ef13-2a33-4d9f-929f-03fb89596187 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.713095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44ef13-2a33-4d9f-929f-03fb89596187 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.713516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44ef13-2a33-4d9f-929f-03fb89596187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.736385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4a44ef13-2a33-4d9f-929f-03fb89596187 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:47.738196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4a44ef13-2a33-4d9f-929f-03fb89596187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:47.797279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b32651-78f6-44fa-9bc7-2c87da3057dd None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:47.850760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:47.851029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:47.851132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:47.851410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:47.852148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:47.852460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:47.861866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 274/562] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:47.862092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:47.862189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44ef13-2a33-4d9f-929f-03fb89596187 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:47.877862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 289/563] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:47.879194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:47.893638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:48.003574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:48.007599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44ef13-2a33-4d9f-929f-03fb89596187 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.019438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45ba72-d347-447b-b11f-d44889110995 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.064647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.065531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.065856 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.066138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.066905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:48.067222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.068740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:48.076265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.077117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.077391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.077794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.085110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 275/564] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:48.085110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.097598 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.099669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:48.117903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:48.120071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 290/565] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:48.121930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.213485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:48.264411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d014bc-0447-4364-98c4-4f97d187ec50 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.266123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefeee8-3960-4a33-b2c1-914a846ffbed None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.313204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce88f3-19a6-490a-8886-6d506aff6782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.314053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce88f3-19a6-490a-8886-6d506aff6782 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.314324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce88f3-19a6-490a-8886-6d506aff6782 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.314674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce88f3-19a6-490a-8886-6d506aff6782 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.318091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.319075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.319455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.319737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.327764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce88f3-19a6-490a-8886-6d506aff6782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:48.330425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.331929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-53ce88f3-19a6-490a-8886-6d506aff6782 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:48.335717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:48.336810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:48.338183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:48.342531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 291/566] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:48.343528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce88f3-19a6-490a-8886-6d506aff6782 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:48.348862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:48.349896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:48.350827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 276/567] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:48.351417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.390164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:48.479148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce88f3-19a6-490a-8886-6d506aff6782 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.494702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8330eaa-9821-4e0b-880f-bf5d2cda1c8c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.517180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:48.531265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.532442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.532895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.533350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.547467 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.548590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.549054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.549512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.550920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:48.551406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.552488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.554046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:48.555028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:48.583414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 277/568] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:48.583414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:48.583697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:48.584532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:48.585471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:48.585471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:48.585471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:48.585471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:48.585471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 292/569] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:48.585471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.705839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:48.730758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbc431-40d5-4ed6-9067-4c11a077d015 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.742759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc4230-55a8-44c3-a23a-b0886e7dbb90 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.771587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.772152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.772439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.772714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.773501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:48.773838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.775376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:48.785206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 278/570] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:48.785593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.786095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.786223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.786525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.786810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.787656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:48.787975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.789441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:48.811010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 293/571] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:48.811891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.861520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:48.916432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52846720-bf99-4aee-8de5-2cd86b7d7763 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.940070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065804b9-ee5f-4a2a-a2c8-9d221c383c12 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:48.953032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79fe99-2a69-45c1-9ba7-a094bf601953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.953712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79fe99-2a69-45c1-9ba7-a094bf601953 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.954012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79fe99-2a69-45c1-9ba7-a094bf601953 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.954285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79fe99-2a69-45c1-9ba7-a094bf601953 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.955121 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79fe99-2a69-45c1-9ba7-a094bf601953 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=203e654f95bf45e683b0ccc4ba687bfc) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:48.955424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79fe99-2a69-45c1-9ba7-a094bf601953 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.956905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79fe99-2a69-45c1-9ba7-a094bf601953 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:48.965606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 279/572] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/role_assignments?user.id=203e654f95bf45e683b0ccc4ba687bfc&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:48.967057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61326f-fa81-4759-be5e-5aaf523a399c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.967346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61326f-fa81-4759-be5e-5aaf523a399c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.967633 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61326f-fa81-4759-be5e-5aaf523a399c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.968980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 280/573] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:48.969759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61326f-fa81-4759-be5e-5aaf523a399c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:48.990390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:48.991509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:48.992001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:48.992470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:48.993117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:48.994744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:48.995161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:48.996761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:49.011569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 294/574] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:49.012456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:49.113336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61326f-fa81-4759-be5e-5aaf523a399c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:49.140186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:49.148820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d410d25-a4cc-4a6d-b5d5-f8d62467878b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:49.165211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec5768-b1e7-4952-86f7-f275b6d15c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:49.166528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec5768-b1e7-4952-86f7-f275b6d15c31 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:49.167056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec5768-b1e7-4952-86f7-f275b6d15c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:49.167424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec5768-b1e7-4952-86f7-f275b6d15c31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:49.181140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec5768-b1e7-4952-86f7-f275b6d15c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:49.183118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec5768-b1e7-4952-86f7-f275b6d15c31 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-bcec5768-b1e7-4952-86f7-f275b6d15c31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:49.202445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:49.204305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:49.206180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:49.206180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:49.206180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:49.206180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:49.206641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:49.206641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:49.206641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:49.206641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:49.208514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 281/575] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:49.209733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1eda5-758e-423a-99af-840693d19adb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:49.210298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1eda5-758e-423a-99af-840693d19adb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:49.210298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1eda5-758e-423a-99af-840693d19adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:49.221883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:49.223419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:49.231944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fcf1eda5-758e-423a-99af-840693d19adb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:49.233191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fcf1eda5-758e-423a-99af-840693d19adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:49.251224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:49.252233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:49.253252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:49.253252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:49.253252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:49.253252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:49.257860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 295/576] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:49.259072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:49.328090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:49.342836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 282/577] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:49.344731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1eda5-758e-423a-99af-840693d19adb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:49.450609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80fea80-5dd6-40d2-b9eb-bf4c1a4193b7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:49.503487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:49.504438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:49.505006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:49.505399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:49.521287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:49.523730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:49.535345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1eda5-758e-423a-99af-840693d19adb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:49.544071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:49.546254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:05:49.548401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:49.550471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 296/578] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:49.553515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:49.579784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:49.589506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:49.590320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:49.590677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:49.591017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:49.591864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:49.592243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:49.593784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:49.607227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 283/579] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:49.613023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:49.691810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:49.752082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9bc9-2a6d-4466-a2bc-de95782fb42b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:49.783123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400382-7b05-48e9-b248-9c08a60ba15b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:49.810949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:49.812241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:49.812718 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:49.813160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:49.814404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:49.814915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:49.816463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:49.841379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 297/580] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:49.841379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:49.844480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96b8b-8274-4e43-b4ac-36e8b7737d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:49.845545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96b8b-8274-4e43-b4ac-36e8b7737d91 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:49.845891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96b8b-8274-4e43-b4ac-36e8b7737d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:49.846161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96b8b-8274-4e43-b4ac-36e8b7737d91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:49.862167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96b8b-8274-4e43-b4ac-36e8b7737d91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-29b96b8b-8274-4e43-b4ac-36e8b7737d91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:49.870971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:49.872031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:49.872942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 284/581] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:49.873722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96b8b-8274-4e43-b4ac-36e8b7737d91 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:49.944440 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:49.994772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a070e-bbbe-4e07-9ae7-a3570ffc5089 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.015718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96b8b-8274-4e43-b4ac-36e8b7737d91 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.030533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:50.061806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.062530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.062893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.063333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.064372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:50.064772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.066347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.085196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.087108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.087986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.088533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 298/582] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 18 14:05:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:50.089829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.094227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.107592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.110424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:50.135858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:50.137387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 285/583] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:50.146085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.219884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4386c31-4a6c-4d82-ae4b-f70fe56d3e90 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.255897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:50.267072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.267797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.268073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.268345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.276830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.278623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.286888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f2d89-8bf3-45c9-aed5-1bb7af249cf0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:50.297782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:50.298914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:50.299836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 299/584] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:50.300334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.336862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.338221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.338789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.339147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.339915 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:50.340230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.341692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.356616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 286/585] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:50.359210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.400920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:50.432472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666e579-a571-4a50-8295-aff52ea5eebd None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.475585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.476399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.476867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.477475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.491850 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.492169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233651e2-59fb-4e83-b32c-f7161b4dcd0f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.493832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:50.515106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:50.516269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:50.517252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:50.517252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:50.517252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:50.517252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:50.517252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 300/586] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:50.518276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.540295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.540959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.541210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.542011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.542884 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:50.543197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.544659 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.565693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 287/587] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:50.566575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.578156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:50.677837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c1451-4efb-4e7e-836b-ed5d42f09b36 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.696163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:50.709485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c81d84-5555-4fc8-8ec2-ba72aa09df24 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.719842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.720511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.720791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.721067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.721964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:50.722307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.723819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.736908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 301/588] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:50.739204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.751128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.752281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.752752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.753180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.765579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.767084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:50.786483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:50.793299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:50.794551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:50.794551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:50.794551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:50.794551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:50.794551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 288/589] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:50.794551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.863999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:50.870471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b46b8-c600-48ec-82c6-1962ecebe6c8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.912080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbb30-719a-4fa7-8648-0981d2ae4d97 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:50.913622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.914510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.914790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.915064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.915936 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:50.916262 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.917877 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.935563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 302/590] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:50.935563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:50.955125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:50.955929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:50.956260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:50.956536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:50.969499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:50.971040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:50.972897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:50.989619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:50.990809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:50.998160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:50.998160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:50.998160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:50.998160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:50.998160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 289/591] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:50.998160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:51.067031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcde0cd-dfbe-4a76-a8db-6b46478a845c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:51.116374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a555e20-7c97-4357-aacf-5f1f62551800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:51.116710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a555e20-7c97-4357-aacf-5f1f62551800 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.117192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a555e20-7c97-4357-aacf-5f1f62551800 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.117917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a555e20-7c97-4357-aacf-5f1f62551800 None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/4fffb91636744fc39886bdf568317ab5/roles/cc93d9b4856a4b1cbf172a716c54860a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.121322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c02e9-9ecb-4acb-a8da-c44d28074eab None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:51.142711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a555e20-7c97-4357-aacf-5f1f62551800 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=4fffb91636744fc39886bdf568317ab5, role_id=cc93d9b4856a4b1cbf172a716c54860a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:51.144174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a555e20-7c97-4357-aacf-5f1f62551800 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:51.159698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 303/592] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:05:50 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/4fffb91636744fc39886bdf568317ab5/roles/cc93d9b4856a4b1cbf172a716c54860a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:51.164509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:51.165391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.166149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.166278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.167059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:51.167346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:51.168787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:51.171376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a555e20-7c97-4357-aacf-5f1f62551800 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:51.186312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:51.192425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 290/593] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:51.197473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:51.280167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:51.320848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a555e20-7c97-4357-aacf-5f1f62551800 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:51.343294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d61637-3abc-471d-8a68-45b2ff087819 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:51.367711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:51.368895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.369178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.369462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.377158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:51.378669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:51.393138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:51.394246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 304/594] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:51.395852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:51.402473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166f5cc-5778-4f43-abfb-207ced4af088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:51.403178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166f5cc-5778-4f43-abfb-207ced4af088 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.403463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166f5cc-5778-4f43-abfb-207ced4af088 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.403723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166f5cc-5778-4f43-abfb-207ced4af088 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.404485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166f5cc-5778-4f43-abfb-207ced4af088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:51.404791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166f5cc-5778-4f43-abfb-207ced4af088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:51.406296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d166f5cc-5778-4f43-abfb-207ced4af088 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:51.420228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 291/595] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:51.421536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d5b2-34c3-426b-9328-9867f30a0368 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.421879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d5b2-34c3-426b-9328-9867f30a0368 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.422161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930d5b2-34c3-426b-9328-9867f30a0368 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.423595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 292/596] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:51.424155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930d5b2-34c3-426b-9328-9867f30a0368 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:51.484638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:51.522490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8716368-404c-49c3-9596-9a6798e1c7c2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:51.549065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930d5b2-34c3-426b-9328-9867f30a0368 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:51.567636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:51.568118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ba409-c0a3-493a-8170-07e9b7614405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:51.568959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ba409-c0a3-493a-8170-07e9b7614405 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.569289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ba409-c0a3-493a-8170-07e9b7614405 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.569588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ba409-c0a3-493a-8170-07e9b7614405 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.585397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ba409-c0a3-493a-8170-07e9b7614405 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:51.587915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ba409-c0a3-493a-8170-07e9b7614405 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:51.597564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:51.598463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.598712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.598969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.599772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:51.600221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:51.601563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:51.620679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 293/597] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:51.620679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-983ba409-c0a3-493a-8170-07e9b7614405 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:51.620975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:51.621934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 305/598] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7898a2-b12d-4171-970e-2a9ad045999d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.622886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7898a2-b12d-4171-970e-2a9ad045999d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.623613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7898a2-b12d-4171-970e-2a9ad045999d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.644370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8c7898a2-b12d-4171-970e-2a9ad045999d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:51.645725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8c7898a2-b12d-4171-970e-2a9ad045999d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:51.738302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 306/599] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:51.739373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7898a2-b12d-4171-970e-2a9ad045999d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:51.764213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036890b-ce1d-4510-8642-214c24bf6a75 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:51.800927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:51.801766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.802049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.802323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.809032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:51.810471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:51.812063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:51.827853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:51.828888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 294/600] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:51.830584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:51.875798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:51.876369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7898a2-b12d-4171-970e-2a9ad045999d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:51.921351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73967c73-47c9-439c-b95b-bcb97539a903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:51.922432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73967c73-47c9-439c-b95b-bcb97539a903 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:51.922725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73967c73-47c9-439c-b95b-bcb97539a903 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:51.923002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73967c73-47c9-439c-b95b-bcb97539a903 None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/812ba8119a5a4297b409525861ec8118/roles/cc93d9b4856a4b1cbf172a716c54860a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:51.943129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73967c73-47c9-439c-b95b-bcb97539a903 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=812ba8119a5a4297b409525861ec8118, role_id=cc93d9b4856a4b1cbf172a716c54860a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:51.944595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73967c73-47c9-439c-b95b-bcb97539a903 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:51.961347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 307/601] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:05:51 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/812ba8119a5a4297b409525861ec8118/roles/cc93d9b4856a4b1cbf172a716c54860a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:51.962327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73967c73-47c9-439c-b95b-bcb97539a903 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:52.000327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b19bc7-a858-4a0a-9746-49e4159d274d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.050717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.051487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.051818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.052097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.053510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:52.053913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:52.055534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:52.086169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 295/602] 10.222.0.71 () {56 vars in 1139 bytes} [Mon Jul 18 14:05:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:52.088377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:52.128229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73967c73-47c9-439c-b95b-bcb97539a903 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.145376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:52.186383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace7f31-9384-4174-bd0b-f503bc504580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.187372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace7f31-9384-4174-bd0b-f503bc504580 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.188069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace7f31-9384-4174-bd0b-f503bc504580 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.188365 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace7f31-9384-4174-bd0b-f503bc504580 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.201529 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace7f31-9384-4174-bd0b-f503bc504580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7ace7f31-9384-4174-bd0b-f503bc504580 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:52.209421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:52.211400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:52.213103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 308/603] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:52.215429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace7f31-9384-4174-bd0b-f503bc504580 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:52.242517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:52.256952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983d026-e708-4372-bbba-967dc228efa0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.298140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.298719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.299006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.299341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.300179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:52.300502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:52.302030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:52.321520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 296/604] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:52.326613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:52.373234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace7f31-9384-4174-bd0b-f503bc504580 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.414360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.416580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.417162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.417692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.432303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:52.434206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:52.445855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141a7cc-1549-4bc9-a2ee-b9e8a4378372 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:52.455575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:52.456492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:52.457363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:52.457363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:52.457363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:52.457363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:52.464953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 309/605] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:52.466277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:52.484309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.484967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.485259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.485579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.486379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:52.486737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:52.488194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:52.497888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:52.498153 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 297/606] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:52.499001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:52.516649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:52.590270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20cbe8-8b56-4055-a962-bb8e7f3e20bb None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.609396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab85912-d748-46f9-a588-de4ec9152286 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.646583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f695686-8455-4154-972d-85dc233d3016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.648368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f695686-8455-4154-972d-85dc233d3016 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.648761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f695686-8455-4154-972d-85dc233d3016 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.649160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f695686-8455-4154-972d-85dc233d3016 None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/d661615b2dd94d9db6183f580ab0fee6/roles/cc93d9b4856a4b1cbf172a716c54860a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.661449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.662467 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.662912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.663327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.664198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:52.664702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:52.666581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:52.670657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f695686-8455-4154-972d-85dc233d3016 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=d661615b2dd94d9db6183f580ab0fee6, role_id=cc93d9b4856a4b1cbf172a716c54860a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:52.672293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f695686-8455-4154-972d-85dc233d3016 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:52.683648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 298/607] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:52.684868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:52.691316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 310/608] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:05:52 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/d661615b2dd94d9db6183f580ab0fee6/roles/cc93d9b4856a4b1cbf172a716c54860a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:52.697874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f695686-8455-4154-972d-85dc233d3016 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:52.838484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:52.843686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:52.893871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebd625-e66f-47f5-80b3-6f8625dfdc71 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.924076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f695686-8455-4154-972d-85dc233d3016 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:52.947382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.948759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.949251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.949773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.966867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:52.969648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:52.976187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:52.976991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:52.977271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:52.977540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:52.986668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:52.988135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:52.995260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:52.996193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:52.997066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:52.997066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:52.997066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:52.997066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:53.003550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 299/609] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:53.005629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:53.009067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:53.009986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:53.010861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:53.010861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:53.010861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:53.010861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:53.010861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:53.010861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:53.010861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:53.010861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:53.011955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 311/610] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:53.018431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:53.137686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f35bc-90c7-4b90-bdb9-9c74c6dacc98 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:53.167651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:53.173249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:53.173964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:53.174302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:53.174528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:53.175324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:53.175637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:53.177360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:53.190627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76828c-71fe-427b-9d74-fbdfb3197ec4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:53.192064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 312/611] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:53.201048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:53.263433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:53.309869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd887df-c774-4a4f-ba22-74367b5f9fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:53.310567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd887df-c774-4a4f-ba22-74367b5f9fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:53.310859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd887df-c774-4a4f-ba22-74367b5f9fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:53.311134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd887df-c774-4a4f-ba22-74367b5f9fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:53.311878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd887df-c774-4a4f-ba22-74367b5f9fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:53.312187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd887df-c774-4a4f-ba22-74367b5f9fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:53.313703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd887df-c774-4a4f-ba22-74367b5f9fb4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:53.320576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e2d38-aaf3-4084-bef9-223909d9fb5e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:53.334586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 300/612] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:53.336562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bf512-107a-4416-8ced-d14fd64a033d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:53.336855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bf512-107a-4416-8ced-d14fd64a033d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:53.337229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bf512-107a-4416-8ced-d14fd64a033d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:53.339164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 301/613] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:53.341370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bf512-107a-4416-8ced-d14fd64a033d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:53.376653 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6b9a-c826-4910-bff9-a9fb4ad721e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:53.377569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6b9a-c826-4910-bff9-a9fb4ad721e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:53.377891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6b9a-c826-4910-bff9-a9fb4ad721e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:53.378193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6b9a-c826-4910-bff9-a9fb4ad721e4 None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/1f1fe084c3164e50bf5d904350925f6a/roles/cc93d9b4856a4b1cbf172a716c54860a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:53.394724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6b9a-c826-4910-bff9-a9fb4ad721e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=1f1fe084c3164e50bf5d904350925f6a, role_id=cc93d9b4856a4b1cbf172a716c54860a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:53.396829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6b9a-c826-4910-bff9-a9fb4ad721e4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:53.414901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 313/614] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:05:53 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/1f1fe084c3164e50bf5d904350925f6a/roles/cc93d9b4856a4b1cbf172a716c54860a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:53.416576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2a91b-d162-4f52-b86d-7febd1a0ba0e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:53.417293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2a91b-d162-4f52-b86d-7febd1a0ba0e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:53.417700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2a91b-d162-4f52-b86d-7febd1a0ba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:53.441690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c2e2a91b-d162-4f52-b86d-7febd1a0ba0e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:53.443609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c2e2a91b-d162-4f52-b86d-7febd1a0ba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:53.496867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bf512-107a-4416-8ced-d14fd64a033d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:53.516652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:53.574838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:53.575667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:53.576007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:53.576287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:53.585957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:53.587883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:53.604447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:53.605706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:53.607056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 302/615] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:05:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:53.611930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:53.614470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 314/616] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:53.661910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2a91b-d162-4f52-b86d-7febd1a0ba0e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:53.663753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:53.734021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b347ae-eecd-4003-ac71-066c0d28f9ab None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:53.798731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2a91b-d162-4f52-b86d-7febd1a0ba0e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:53.820413 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:53.821112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:53.821463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:53.821752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:53.822761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:53.823075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:53.824560 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:53.841096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7f5c3-4b64-430d-ac92-7081b24dc360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:53.841975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7f5c3-4b64-430d-ac92-7081b24dc360 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:53.842257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7f5c3-4b64-430d-ac92-7081b24dc360 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:53.842537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe7f5c3-4b64-430d-ac92-7081b24dc360 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:53.857218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe7f5c3-4b64-430d-ac92-7081b24dc360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-abe7f5c3-4b64-430d-ac92-7081b24dc360 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:53.865786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:53.866914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:53.867816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:53.870409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 303/617] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 18 14:05:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:53.872409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 315/618] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:05:53.874011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:53.874213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7f5c3-4b64-430d-ac92-7081b24dc360 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:53.947113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:54.105843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe7f5c3-4b64-430d-ac92-7081b24dc360 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:54.144010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983176fc-890e-48dc-8aa4-8fd0700d60d9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:54.150191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:54.176154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:54.176858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.177154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.177404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:54.178469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:54.179642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:54.181211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:54.200592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 316/619] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:54.210894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:54.237626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:54.238491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.238781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.239055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:54.264012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:54.265495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:54.299191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:54.300944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:54.303328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:54.303328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:54.303328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:54.303328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:54.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 304/620] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:54.318149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:54.461064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:54.484645 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67a2fcb-07c8-44d2-9625-53a9a9c48e8c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:54.511886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5428b3-3aca-4dba-9850-6a0a001ecd70 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:54.549103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a3a98-5075-4838-85d3-214d16b6cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:54.550602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3a98-5075-4838-85d3-214d16b6cd9b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.551072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3a98-5075-4838-85d3-214d16b6cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.551681 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a3a98-5075-4838-85d3-214d16b6cd9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:54.559160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d259acd-1b50-4289-8498-56919451c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:54.559848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259acd-1b50-4289-8498-56919451c394 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.560141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259acd-1b50-4289-8498-56919451c394 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.560391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d259acd-1b50-4289-8498-56919451c394 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:54.561149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259acd-1b50-4289-8498-56919451c394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:54.561440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259acd-1b50-4289-8498-56919451c394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:54.562863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d259acd-1b50-4289-8498-56919451c394 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:54.572260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a3a98-5075-4838-85d3-214d16b6cd9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:54.575354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a3a98-5075-4838-85d3-214d16b6cd9b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:54.579164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 305/621] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:05:54.580978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cb165-b875-4841-9320-d39b9190a2d2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.581268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cb165-b875-4841-9320-d39b9190a2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.581545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cb165-b875-4841-9320-d39b9190a2d2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:54.587070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 306/622] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:54.589266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cb165-b875-4841-9320-d39b9190a2d2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-af0a3a98-5075-4838-85d3-214d16b6cd9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:54.600962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:54.601948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:54.603629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:54.603629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:54.603629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:54.603629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:54.603629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 317/623] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:54.604290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ee2d-23cc-4278-9925-cefc21a33e5d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.605502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ee2d-23cc-4278-9925-cefc21a33e5d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.605502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193ee2d-23cc-4278-9925-cefc21a33e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:54.610514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:54.630308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c193ee2d-23cc-4278-9925-cefc21a33e5d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:54.631572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c193ee2d-23cc-4278-9925-cefc21a33e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:54.722091 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 318/624] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:54.722979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193ee2d-23cc-4278-9925-cefc21a33e5d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:54.741925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cb165-b875-4841-9320-d39b9190a2d2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:54.791499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:54.792833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.793298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.794689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:54.805582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:54.807214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:54.827396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:54.828403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:54.829371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 307/625] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:54.829909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:54.872059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:54.910751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193ee2d-23cc-4278-9925-cefc21a33e5d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:54.921309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:54.949193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2decc-0b76-4eb6-af65-7bd50af16f4c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:54.957031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:54.958209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.958696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.959168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:54.960404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:54.961125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:54.963701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:54.979565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 319/626] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:54.981073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:54.990836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8dde-b867-44e2-99dc-becc762fff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:54.991685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8dde-b867-44e2-99dc-becc762fff30 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:54.991969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8dde-b867-44e2-99dc-becc762fff30 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:54.992249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd8dde-b867-44e2-99dc-becc762fff30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.004788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bd8dde-b867-44e2-99dc-becc762fff30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a9bd8dde-b867-44e2-99dc-becc762fff30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:55.014212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:55.015252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:55.016124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:55.020543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 308/627] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:55.021462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8dde-b867-44e2-99dc-becc762fff30 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:55.107687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf767a-f0b1-4d4e-a8af-5dee61341256 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.124685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd8dde-b867-44e2-99dc-becc762fff30 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.147566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.148697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.149225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.149778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.151600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:55.152236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:55.154269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:55.156272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:55.160406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.161401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.161693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.161979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.165700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:55.172845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 320/628] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:55.173252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:55.174254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:55.174918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:55.190836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:55.191905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:55.193493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 309/629] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:05:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:55.194602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:55.296150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7bb71e-192e-4b69-aff3-877ec26fd874 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.298765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19959b11-d030-4629-9e40-b17d35f37d21 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.331498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf54c5c-ea66-4d5e-a2d9-4c5ff25b8d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.332395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf54c5c-ea66-4d5e-a2d9-4c5ff25b8d1e None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.332683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf54c5c-ea66-4d5e-a2d9-4c5ff25b8d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.332963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf54c5c-ea66-4d5e-a2d9-4c5ff25b8d1e None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/203e654f95bf45e683b0ccc4ba687bfc/roles/cc93d9b4856a4b1cbf172a716c54860a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.340874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.342025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.342590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.343061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.344023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:55.344526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:55.346358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:55.349054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf54c5c-ea66-4d5e-a2d9-4c5ff25b8d1e None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=203e654f95bf45e683b0ccc4ba687bfc, role_id=cc93d9b4856a4b1cbf172a716c54860a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:55.350539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf54c5c-ea66-4d5e-a2d9-4c5ff25b8d1e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:55.358835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 321/630] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:55.360105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:55.370865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 310/631] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:05:55 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/203e654f95bf45e683b0ccc4ba687bfc/roles/cc93d9b4856a4b1cbf172a716c54860a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:05:55.371554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf54c5c-ea66-4d5e-a2d9-4c5ff25b8d1e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:55.426849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:55.450389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:55.517492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf54c5c-ea66-4d5e-a2d9-4c5ff25b8d1e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.555189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5507f5e5-0486-4976-8678-cd33249d2663 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.583816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9248a63-bf5c-4272-8590-c769a0dea1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.585430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9248a63-bf5c-4272-8590-c769a0dea1ee None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.585430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9248a63-bf5c-4272-8590-c769a0dea1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.585430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9248a63-bf5c-4272-8590-c769a0dea1ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.601620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9248a63-bf5c-4272-8590-c769a0dea1ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:55.602601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9248a63-bf5c-4272-8590-c769a0dea1ee None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:55.625120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955b1dd-9391-4a2f-94d5-4940593b3f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.626008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955b1dd-9391-4a2f-94d5-4940593b3f2c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a9248a63-bf5c-4272-8590-c769a0dea1ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:55.626120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:55.627031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:55.627906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:55.627906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:55.627906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:55.627906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:55.628135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955b1dd-9391-4a2f-94d5-4940593b3f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.628135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955b1dd-9391-4a2f-94d5-4940593b3f2c None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.628135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955b1dd-9391-4a2f-94d5-4940593b3f2c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:55.630889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955b1dd-9391-4a2f-94d5-4940593b3f2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:55.630889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955b1dd-9391-4a2f-94d5-4940593b3f2c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:55.632102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 311/632] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:55.633464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e44da8-8aac-4352-a8d6-f63459506077 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.633778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e44da8-8aac-4352-a8d6-f63459506077 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.634099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e44da8-8aac-4352-a8d6-f63459506077 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.635838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 312/633] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:55.636649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e44da8-8aac-4352-a8d6-f63459506077 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:55.661347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 322/634] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:05:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:55.662692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b384-9bc7-4e75-a3cb-b7f8eaa004f6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.662987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b384-9bc7-4e75-a3cb-b7f8eaa004f6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.663259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397b384-9bc7-4e75-a3cb-b7f8eaa004f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.681537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b397b384-9bc7-4e75-a3cb-b7f8eaa004f6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:55.682817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b397b384-9bc7-4e75-a3cb-b7f8eaa004f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:55.767999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 323/635] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:55.769326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397b384-9bc7-4e75-a3cb-b7f8eaa004f6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:55.772148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:55.786523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e44da8-8aac-4352-a8d6-f63459506077 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.823354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.823949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.824313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.824587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.825315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:55.825627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:55.827069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:55.827569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:55.840058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 313/636] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:55.841042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:55.887965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397b384-9bc7-4e75-a3cb-b7f8eaa004f6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.926740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8634d20-bc0d-412b-bf44-e137b729771a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.927485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8634d20-bc0d-412b-bf44-e137b729771a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.927777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8634d20-bc0d-412b-bf44-e137b729771a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.928176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8634d20-bc0d-412b-bf44-e137b729771a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:55.957627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cae53-47e9-45c5-9303-ee41837eeccf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:55.997571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da40c-88ea-4332-a0ca-0556b32af5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:55.998696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da40c-88ea-4332-a0ca-0556b32af5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:55.999105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da40c-88ea-4332-a0ca-0556b32af5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:55.999512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823da40c-88ea-4332-a0ca-0556b32af5f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.011274 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8634d20-bc0d-412b-bf44-e137b729771a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:56.012872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8634d20-bc0d-412b-bf44-e137b729771a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:56.015159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823da40c-88ea-4332-a0ca-0556b32af5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-823da40c-88ea-4332-a0ca-0556b32af5f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:56.026650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:56.027732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:56.028587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:56.029247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 314/637] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:56.029850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da40c-88ea-4332-a0ca-0556b32af5f9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.055772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:56.074107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:56.159696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 324/638] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:56.159696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8634d20-bc0d-412b-bf44-e137b729771a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.173965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823da40c-88ea-4332-a0ca-0556b32af5f9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:56.222188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4eb59-c602-4428-b30a-ffe30df3f608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:56.223908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4eb59-c602-4428-b30a-ffe30df3f608 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.224362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4eb59-c602-4428-b30a-ffe30df3f608 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.225066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4eb59-c602-4428-b30a-ffe30df3f608 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.235871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4eb59-c602-4428-b30a-ffe30df3f608 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:56.237463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4eb59-c602-4428-b30a-ffe30df3f608 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-81e4eb59-c602-4428-b30a-ffe30df3f608 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:56.253382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:56.255203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:05:56.256381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:56.257970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 315/639] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:56.259743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a898b-35d9-4c2c-877c-965edda90b40 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.260022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a898b-35d9-4c2c-877c-965edda90b40 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.260304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a898b-35d9-4c2c-877c-965edda90b40 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.262599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 316/640] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:56.267872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a898b-35d9-4c2c-877c-965edda90b40 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.292722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8634d20-bc0d-412b-bf44-e137b729771a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:56.333966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:56.334649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.335012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.335622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.336955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:56.338496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:56.340946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:56.355466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 325/641] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:56.356327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.366583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:56.377541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:56.402114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a898b-35d9-4c2c-877c-965edda90b40 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:56.439249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f9e9f-1dea-42fb-a813-a595a4111a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:56.440351 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f9e9f-1dea-42fb-a813-a595a4111a34 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.440793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f9e9f-1dea-42fb-a813-a595a4111a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.441206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f9e9f-1dea-42fb-a813-a595a4111a34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.454245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f9e9f-1dea-42fb-a813-a595a4111a34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:56.456037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f9e9f-1dea-42fb-a813-a595a4111a34 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:56.476594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424391f0-67e5-4f89-b2a5-5158dc5ecf81 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-cd7f9e9f-1dea-42fb-a813-a595a4111a34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:56.478617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:56.479941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:56.480848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:56.480848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:56.480848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:56.480848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:56.480848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 317/642] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:56.481136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036521-bd2d-47b8-b6fd-c72d630160b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.481679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036521-bd2d-47b8-b6fd-c72d630160b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.481679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd036521-bd2d-47b8-b6fd-c72d630160b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.501054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fd036521-bd2d-47b8-b6fd-c72d630160b0 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:56.502317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fd036521-bd2d-47b8-b6fd-c72d630160b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:56.521983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:56.522637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.522942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.523189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.524045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:56.524346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:56.525851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:56.544416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 326/643] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:56.545444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.581776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 318/644] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:56.587016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036521-bd2d-47b8-b6fd-c72d630160b0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.617817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:56.661410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:56.664002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdf72b-6839-4e30-9115-a59c3057b3e0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:56.706516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:56.707331 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.707611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.707880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.713978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd036521-bd2d-47b8-b6fd-c72d630160b0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:56.719175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:56.720663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:56.739200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:56.740172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:56.741203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:56.741203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:56.741203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:56.741203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:56.741203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 327/645] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:56.741203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.759647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:56.760286 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.760543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.760846 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.761785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:56.761994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:56.763326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:56.775346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 319/646] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:56.780699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.851849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f0f6a-7fef-4eba-bab5-fe54800e6873 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:56.875992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:56.896193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edf383-af8f-4a9f-add2-923c6d98c820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:56.896554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0274f02-9f85-4f9e-b50b-761e07bb5bd0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:56.896882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edf383-af8f-4a9f-add2-923c6d98c820 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.897165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edf383-af8f-4a9f-add2-923c6d98c820 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.897435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edf383-af8f-4a9f-add2-923c6d98c820 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.927740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:56.946713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc1674-4d3a-40ba-abe9-434443eab64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:56.947640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc1674-4d3a-40ba-abe9-434443eab64a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:56.947929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc1674-4d3a-40ba-abe9-434443eab64a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:56.948208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc1674-4d3a-40ba-abe9-434443eab64a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:56.960374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc1674-4d3a-40ba-abe9-434443eab64a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2ccc1674-4d3a-40ba-abe9-434443eab64a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:56.969277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:56.970363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:56.971222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 320/647] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:56.972645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc1674-4d3a-40ba-abe9-434443eab64a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:56.982335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edf383-af8f-4a9f-add2-923c6d98c820 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:56.983820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edf383-af8f-4a9f-add2-923c6d98c820 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.093564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc1674-4d3a-40ba-abe9-434443eab64a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.137550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 328/648] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:57.137550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edf383-af8f-4a9f-add2-923c6d98c820 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.142938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09672733-b105-4685-a008-616b095a8172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.143795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09672733-b105-4685-a008-616b095a8172 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.144210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09672733-b105-4685-a008-616b095a8172 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.144300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09672733-b105-4685-a008-616b095a8172 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.145103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09672733-b105-4685-a008-616b095a8172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:57.145480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09672733-b105-4685-a008-616b095a8172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.147141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09672733-b105-4685-a008-616b095a8172 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.159712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:57.160801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 321/649] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:57.161904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09672733-b105-4685-a008-616b095a8172 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.209941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:57.257910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edf383-af8f-4a9f-add2-923c6d98c820 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.275945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09672733-b105-4685-a008-616b095a8172 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.296669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.297722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.298067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.298443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.306881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.309090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.315264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.315956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.316242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.316516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.317304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:57.317592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.319012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:57.324124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:57.325136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:57.326098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 329/650] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:57.327295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.338529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 322/651] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:57.339845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.425760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:57.453594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0349c9-4206-482c-94d3-bb111ff42875 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.457570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1c1b8-f403-4985-a91a-69e81ee5d793 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.459216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:57.497695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.498790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.499217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.499647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.499850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.500672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.501224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.510113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.511590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.512632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.514340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:57.527061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:57.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:57.529173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 323/652] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:57.529840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:57.535496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:57.536657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:57.537728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:57.537728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:57.537728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:57.537728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:57.541558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 330/653] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:57.542538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.647906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b3ce8-c35e-4811-ad98-895694db94d4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.654504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7781a0bd-be45-4bb8-aa8c-9de2a224c231 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.682287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:57.684854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.685792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.686014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.686640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.687470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:57.687793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.689247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.691152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.691953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.692257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.692524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.702816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 324/654] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:57.703921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.709519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.712000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.733213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:57.734017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:57.735139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:57.736235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:57.736235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:57.736235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:57.736235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:57.736235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 331/655] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:57.736235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.823894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff38cd-d3a8-492a-a1f8-5218d4cf0df3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.853932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93151aa-8db7-47d3-921c-ec74c52dcf93 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:57.871328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb392f-15e1-49f4-833b-a431dad0005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.872012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb392f-15e1-49f4-833b-a431dad0005a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.872276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb392f-15e1-49f4-833b-a431dad0005a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.872558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb392f-15e1-49f4-833b-a431dad0005a None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.873491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb392f-15e1-49f4-833b-a431dad0005a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:57.873923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb392f-15e1-49f4-833b-a431dad0005a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.875426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb392f-15e1-49f4-833b-a431dad0005a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.895759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:57.896458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.896826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.897019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.897345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 325/656] 10.222.0.71 () {56 vars in 1137 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:57.903258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:57.903258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:57.903554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd540-e025-4772-bf01-102f1e6edac8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:57.903678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:57.903828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd540-e025-4772-bf01-102f1e6edac8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:57.903828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bd540-e025-4772-bf01-102f1e6edac8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:57.903828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 326/657] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:05:57.903828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bd540-e025-4772-bf01-102f1e6edac8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.916517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 332/658] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:57.917406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:57.967963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:58.010820 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:58.037440 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb5a8b-509b-4fca-814e-c676a8164273 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:58.038174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bd540-e025-4772-bf01-102f1e6edac8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:58.080264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:58.080940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.081214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.081490 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.082360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=4fffb91636744fc39886bdf568317ab5) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:58.082677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:58.084138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:58.085226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba576c-503c-49f1-a695-6c5a6698504d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:58.086136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba576c-503c-49f1-a695-6c5a6698504d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.086277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba576c-503c-49f1-a695-6c5a6698504d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.086490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba576c-503c-49f1-a695-6c5a6698504d None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.087989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba576c-503c-49f1-a695-6c5a6698504d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:58.087989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba576c-503c-49f1-a695-6c5a6698504d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:58.088971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba576c-503c-49f1-a695-6c5a6698504d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:58.091167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 327/659] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/role_assignments?user.id=4fffb91636744fc39886bdf568317ab5&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:58.092015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:58.107898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 333/660] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:58.107898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99afcb-4c04-49c8-bed3-6c84767514b7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.107898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99afcb-4c04-49c8-bed3-6c84767514b7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.107898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99afcb-4c04-49c8-bed3-6c84767514b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.128700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bb99afcb-4c04-49c8-bed3-6c84767514b7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:05:58.130140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bb99afcb-4c04-49c8-bed3-6c84767514b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:05:58.218981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 334/661] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:05:58.219617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99afcb-4c04-49c8-bed3-6c84767514b7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:58.243761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a461dfc-c63f-47b5-a3da-56d605daadca None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:58.266365 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:58.281006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fde42-1ef5-4a7b-99c6-2c2e6df130be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:58.281698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fde42-1ef5-4a7b-99c6-2c2e6df130be None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.281983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fde42-1ef5-4a7b-99c6-2c2e6df130be None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.282256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fde42-1ef5-4a7b-99c6-2c2e6df130be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.299746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:58.354434 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99afcb-4c04-49c8-bed3-6c84767514b7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:58.367359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fde42-1ef5-4a7b-99c6-2c2e6df130be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:58.369281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fde42-1ef5-4a7b-99c6-2c2e6df130be None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:58.391153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:58.391962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.392240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.392514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.399466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:58.401685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:58.417058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:58.418071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:58.418991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:58.418991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:58.418991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:58.418991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:58.418991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:58.418991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:58.418991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:58.418991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:58.421303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 335/662] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:58.422649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:58.477238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 328/663] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:58.478925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fde42-1ef5-4a7b-99c6-2c2e6df130be None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:58.530568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:58.536393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:58.548497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad4be9e-224d-4997-bed6-7808b857b981 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:58.586867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:58.588605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.588890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.589173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.591900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fde42-1ef5-4a7b-99c6-2c2e6df130be None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:58.600075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:58.602208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:05:58.619791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:58.621170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 18 14:05:58.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:05:58.627318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 336/664] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:58.627318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:58.647958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f40a-0ba1-4b4a-ab24-06e9f25832f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:58.648880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f40a-0ba1-4b4a-ab24-06e9f25832f7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.649231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f40a-0ba1-4b4a-ab24-06e9f25832f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.649621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112f40a-0ba1-4b4a-ab24-06e9f25832f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.662761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112f40a-0ba1-4b4a-ab24-06e9f25832f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f112f40a-0ba1-4b4a-ab24-06e9f25832f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:05:58.671801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:58.673588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:05:58.675483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:58.677857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 329/665] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:05:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:58.678910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f40a-0ba1-4b4a-ab24-06e9f25832f7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:58.791167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca0c5c-5e0a-44d5-859e-9ff0c055041d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:58.814028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112f40a-0ba1-4b4a-ab24-06e9f25832f7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:58.817854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:58.836128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:58.837009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.837195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.837426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.838359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:58.838669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:58.840205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:58.855091 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 337/666] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:05:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:58.855091 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:58.864617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:58.865426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:58.865738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:58.866011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:58.869073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:58.878971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:58.880647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:58.905213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:58.906208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:58.907119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:58.907119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:58.907119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:58.907119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:58.909992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 330/667] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:58.911171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:58.995955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988020a-33ac-4c82-968d-f5ab76adc33f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:59.052470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:59.053369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:59.053868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:59.054331 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:59.055771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:59.056297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:59.058836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:59.063626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566938c0-6beb-4f88-8e4c-3c7bca0ad849 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:59.087242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 338/668] 10.222.0.71 () {56 vars in 1139 bytes} [Mon Jul 18 14:05:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:59.088589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:59.116144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:59.116827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:59.117108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:59.117378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:59.118630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:59.119104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:59.120539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:59.132357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 331/669] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:05:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:59.133221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:59.159369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:59.173349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:59.215203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550679d-2b65-4a11-b14a-4daec6aa8b5d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:59.254981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:59.256083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:59.256539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:59.257000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:59.258627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=812ba8119a5a4297b409525861ec8118) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:59.259643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:59.262212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:59.270432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7083d56-7214-4cd3-8963-d6c734bada74 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:59.273257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 339/670] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:05:59 2022] GET /identity/v3/role_assignments?user.id=812ba8119a5a4297b409525861ec8118&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:59.274624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:59.321266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:59.323236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:59.323740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:59.324194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:59.325162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:05:59.325683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:59.327796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:59.346171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 332/671] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:05:59.347683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:59.462667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:59.467507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14b81e-cf43-4303-85d7-3a5963f9c1c4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:59.499200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31977e17-33c7-40c7-b9f5-76ac7e237ee4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:59.520546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:59.529712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224f9b1-4da8-4a2f-a281-a437c98ce3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:59.530768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224f9b1-4da8-4a2f-a281-a437c98ce3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:59.531273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224f9b1-4da8-4a2f-a281-a437c98ce3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:59.531650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224f9b1-4da8-4a2f-a281-a437c98ce3e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:59.550644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:59.551788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:59.552181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:59.552559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:59.570042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:59.573619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:05:59.595784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:59.596762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:59.598088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:05:59.598088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:05:59.598088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:05:59.598088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:59.600384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 333/672] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:59.601682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:59.645341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224f9b1-4da8-4a2f-a281-a437c98ce3e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:59.647812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224f9b1-4da8-4a2f-a281-a437c98ce3e3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:59.735272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b20b-f55d-4988-b03d-c99e391afa2a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:05:59.776763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:59.787738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:05:59.789372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:05:59.790339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:05:59.791031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:05:59.800124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 340/673] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:05:59.800124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224f9b1-4da8-4a2f-a281-a437c98ce3e3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:59.812614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:05:59.815831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:05:59.835617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:05:59.844743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:05:59.846738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:05:59.848893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:05:59.848893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:05:59.848893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:05:59.848893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:05:59.848893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:05:59.848893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:05:59.848893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:05:59.848893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:05:59.852831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 334/674] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:05:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:05:59.854573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:05:59.950946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224f9b1-4da8-4a2f-a281-a437c98ce3e3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.000428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7c752-8277-403e-a7a5-c288b5454a21 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.010568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.012080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.012792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.013443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.014921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:00.015642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:00.018414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:00.034540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 341/675] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:00.035658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:00.043590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.044547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.044932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.045336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.061531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:00.064096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:00.084394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:00.085550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:00.086791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:00.086791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:00.086791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:00.086791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:00.093212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 335/676] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:00.094446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:00.142517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:00.162661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:00.176338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e40a53-1eaa-4249-920a-279a8fa69b46 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.215937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1402da-81dd-4495-9d06-dc05145c9243 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.252110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaecf39-f147-4e02-9da4-8c147f7d73b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.252338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.253328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaecf39-f147-4e02-9da4-8c147f7d73b0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.253759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.253990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaecf39-f147-4e02-9da4-8c147f7d73b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.254406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaecf39-f147-4e02-9da4-8c147f7d73b0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.254590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.255213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.257092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:00.257733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:00.260811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:00.264225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaecf39-f147-4e02-9da4-8c147f7d73b0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:00.265933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaecf39-f147-4e02-9da4-8c147f7d73b0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:00.279614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 342/677] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:00.280610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6eaecf39-f147-4e02-9da4-8c147f7d73b0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:00.281735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:00.282675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:06:00.283658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:00.285965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 336/678] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:00.287456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017c4f6-8ace-4bdb-a940-4ce90d9fafe9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.287745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017c4f6-8ace-4bdb-a940-4ce90d9fafe9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.288025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017c4f6-8ace-4bdb-a940-4ce90d9fafe9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.289454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 337/679] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:00.290105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017c4f6-8ace-4bdb-a940-4ce90d9fafe9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:00.406596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda856b-af8e-4083-bd1c-e169fd58f734 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.411323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017c4f6-8ace-4bdb-a940-4ce90d9fafe9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.412548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:00.452779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.453485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.453898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.454183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.454615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ba66d-55f5-47ff-949c-2626be4b5961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.455027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=d661615b2dd94d9db6183f580ab0fee6) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:00.455343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:00.455736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba66d-55f5-47ff-949c-2626be4b5961 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.456169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba66d-55f5-47ff-949c-2626be4b5961 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.456591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba66d-55f5-47ff-949c-2626be4b5961 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.456801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:00.457595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ba66d-55f5-47ff-949c-2626be4b5961 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:00.458059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ba66d-55f5-47ff-949c-2626be4b5961 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:00.460603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ba66d-55f5-47ff-949c-2626be4b5961 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:00.463827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 338/680] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/role_assignments?user.id=d661615b2dd94d9db6183f580ab0fee6&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:00.464694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:00.497712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 343/681] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:00.497712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5572a2c-4225-4a1f-be96-d0456b10c0fb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.497712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5572a2c-4225-4a1f-be96-d0456b10c0fb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.497712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5572a2c-4225-4a1f-be96-d0456b10c0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.520597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f5572a2c-4225-4a1f-be96-d0456b10c0fb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:00.521858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f5572a2c-4225-4a1f-be96-d0456b10c0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:00.532645 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:00.612116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82360cea-f6c4-4f43-97a9-3a10f5568092 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.621343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 344/682] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:06:00.623109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5572a2c-4225-4a1f-be96-d0456b10c0fb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:00.664145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.665712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.667304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.668146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.670658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:00.671429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:00.674587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:00.704233 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 339/683] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:00.704233 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:00.736243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:00.779436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5572a2c-4225-4a1f-be96-d0456b10c0fb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.819610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.820292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.820292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.820460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.834020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:00.835516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:00.837400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b40ea-3515-450d-af9c-e370bfa151b1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:00.856734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:00.857913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:00.859170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 345/684] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:00.859777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:00.867154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:00.893744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bc654-9417-479f-9a4e-04329d2494c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:00.893744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bc654-9417-479f-9a4e-04329d2494c5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:00.893744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bc654-9417-479f-9a4e-04329d2494c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:00.893744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bc654-9417-479f-9a4e-04329d2494c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:00.911889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bc654-9417-479f-9a4e-04329d2494c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-663bc654-9417-479f-9a4e-04329d2494c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:00.924575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:00.925830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:00.926683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:00.930025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 340/685] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:00.931713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bc654-9417-479f-9a4e-04329d2494c5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:01.017254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda4139-5723-48cf-b5ab-723c5503e5fa None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:01.048374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:01.062734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2bc0d-5d46-4f74-8a73-687371bec4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:01.063726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2bc0d-5d46-4f74-8a73-687371bec4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.064150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2bc0d-5d46-4f74-8a73-687371bec4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.064596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2bc0d-5d46-4f74-8a73-687371bec4f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.070238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bc654-9417-479f-9a4e-04329d2494c5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:01.114338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:01.115046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.115329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.115730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.116513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:01.117510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:01.119747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:01.140445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 341/686] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:01.140445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:01.164240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2bc0d-5d46-4f74-8a73-687371bec4f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:01.165805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2bc0d-5d46-4f74-8a73-687371bec4f7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:01.178139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:01.286483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a42119-4228-4f2f-a30d-cff9ef379deb None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:01.298952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 346/687] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:06:01.300761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2bc0d-5d46-4f74-8a73-687371bec4f7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:01.329433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:01.330333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.330732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.331649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.332709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:01.333215 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:01.334944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:01.348613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 342/688] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:01.349552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:01.362446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:01.451126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2bc0d-5d46-4f74-8a73-687371bec4f7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:01.452886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:01.490145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:01.491016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.491409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.491809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.492778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=1f1fe084c3164e50bf5d904350925f6a) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:01.493824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:01.495284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:01.502278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 347/689] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:01 2022] GET /identity/v3/role_assignments?user.id=1f1fe084c3164e50bf5d904350925f6a&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:01.503318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:01.522260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24771c3a-d690-4564-9f81-38c237685f0d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:01.591801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f7a6c-aa64-4e61-baa7-e84a94a2e4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:01.593485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7a6c-aa64-4e61-baa7-e84a94a2e4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.594176 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7a6c-aa64-4e61-baa7-e84a94a2e4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.594813 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7a6c-aa64-4e61-baa7-e84a94a2e4c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.614752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f7a6c-aa64-4e61-baa7-e84a94a2e4c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:01.618942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f7a6c-aa64-4e61-baa7-e84a94a2e4c9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:01.619158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a2eca-2cfc-4803-8aa4-5a3f5d2f5ccd None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ec9f7a6c-aa64-4e61-baa7-e84a94a2e4c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:01.650144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:01.651108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:01.652015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:01.652015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:01.652015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:01.652015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:01.655631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 343/690] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:01.657416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0cd80-437d-437c-8fbc-ef43f43e0480 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.657962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0cd80-437d-437c-8fbc-ef43f43e0480 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.658557 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0cd80-437d-437c-8fbc-ef43f43e0480 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.660707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 344/691] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:01.661837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0cd80-437d-437c-8fbc-ef43f43e0480 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:01.668219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85b44e-7fbd-4f63-ac18-ec55583779bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:01.669042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b44e-7fbd-4f63-ac18-ec55583779bf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.669373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b44e-7fbd-4f63-ac18-ec55583779bf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.669686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b44e-7fbd-4f63-ac18-ec55583779bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.683744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85b44e-7fbd-4f63-ac18-ec55583779bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:01.685579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85b44e-7fbd-4f63-ac18-ec55583779bf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2e85b44e-7fbd-4f63-ac18-ec55583779bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:01.706325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:01.707267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:01.708293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:01.708293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:01.708293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:01.708293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:01.708293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 348/692] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:01.710007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e82315-bd67-4236-8cc6-4040dbe3082e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.710007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e82315-bd67-4236-8cc6-4040dbe3082e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.710007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e82315-bd67-4236-8cc6-4040dbe3082e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.728218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e8e82315-bd67-4236-8cc6-4040dbe3082e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:01.729429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e8e82315-bd67-4236-8cc6-4040dbe3082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:01.745775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:01.745970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:01.809423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0cd80-437d-437c-8fbc-ef43f43e0480 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:01.829496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 349/693] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:06:01.830435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e82315-bd67-4236-8cc6-4040dbe3082e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:01.855522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:01.856566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:01.857038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:01.857553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:01.858521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:01.859032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:01.860833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:01.877510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 345/694] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:01.882999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:01.999891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e82315-bd67-4236-8cc6-4040dbe3082e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.044493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d127b65-5419-46d5-b807-3cbe960f9dfe None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.062083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.063440 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.063903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.064835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.065686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:02.066205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.068684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.097751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 350/695] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:02.099339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:02.110604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.112345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.113063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.113785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.115295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.117980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.119630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:02.120441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:02.153863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:02.155690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:02.157309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 346/696] 10.222.0.71 () {56 vars in 1129 bytes} [Mon Jul 18 14:06:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:02.158948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:02.280348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25624b-2a18-465e-bef8-f815d0edb94d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.319972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.320815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.321142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.321519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.328966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.330487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.341628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeed29d-1f2a-4805-a3ba-b86de20587d3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:02.345527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:02.347187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 351/697] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:02.348883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:02.391167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.392031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.392298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.392574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.403845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.405379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.425100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:02.425388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:02.426563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:02.427715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:02.431709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 347/698] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:06:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:02.432743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:02.470551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513d653b-c7a0-4ab9-93da-d8e22568d516 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.493545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:02.509172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.510106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.510488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.510866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.511766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:02.512184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.513705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.525807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 352/699] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 18 14:06:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:02.526878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:02.571503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ccb62-db2b-4044-8f45-88a85148760a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.617031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.619347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.619966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.620404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.621509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:02.621984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.623869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.649715 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d9ba9-dc85-44cf-a3b8-fbd57da3a947 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.651227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 348/700] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:02.652394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:02.693017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:02.700489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.701519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.702060 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.702569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.703603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:02.704107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.706458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.738057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 353/701] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:06:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:02.738057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:02.780958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a263239-967c-4f92-a792-2139ca59962c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.783013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:02.822521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.823514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.823810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.824184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.825004 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:02.825414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.826952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.839708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 349/702] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 18 14:06:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:02.840437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:02.871671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaaf9e-d5f9-41bb-81d6-5935f6daf39c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:02.917084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:02.922629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:02.922629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:02.923020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:02.925700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=203e654f95bf45e683b0ccc4ba687bfc) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:02.925700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:02.925700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:02.943641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 354/703] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:02 2022] GET /identity/v3/role_assignments?user.id=203e654f95bf45e683b0ccc4ba687bfc&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:03.031189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2e670-8c78-422f-87df-2cc85a0df154 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:03.085169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca3ffb-48ff-45b8-bd5c-92871ee2bfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:03.086362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3ffb-48ff-45b8-bd5c-92871ee2bfd9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:03.086934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3ffb-48ff-45b8-bd5c-92871ee2bfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:03.087449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3ffb-48ff-45b8-bd5c-92871ee2bfd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:03.137258 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:03.180883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca3ffb-48ff-45b8-bd5c-92871ee2bfd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:03.182890 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca3ffb-48ff-45b8-bd5c-92871ee2bfd9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:03.295728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 350/704] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:06:03.301720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:03.389535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:03.441130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0f422-2cdd-4b27-af25-c37637935927 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:03.481493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc7b08-a163-40b1-8afc-1d40436f597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:03.482284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc7b08-a163-40b1-8afc-1d40436f597a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:03.482615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc7b08-a163-40b1-8afc-1d40436f597a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:03.482892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc7b08-a163-40b1-8afc-1d40436f597a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:03.494999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc7b08-a163-40b1-8afc-1d40436f597a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:03.496548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc7b08-a163-40b1-8afc-1d40436f597a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d9fc7b08-a163-40b1-8afc-1d40436f597a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:03.515636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:03.516593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:03.517495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:03.517495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:03.517495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:03.517495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:03.517495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 355/705] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:03.526024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca3ffb-48ff-45b8-bd5c-92871ee2bfd9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:03.595361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b6374-9507-43d2-a373-7e187a314aac None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:03.595875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b6374-9507-43d2-a373-7e187a314aac None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:03.596459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b6374-9507-43d2-a373-7e187a314aac None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:03.605804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 356/706] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:03.606050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:03.608371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e96dfd-c929-4fa1-a5a5-60a200081227 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:03.608790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e96dfd-c929-4fa1-a5a5-60a200081227 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:03.609297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e96dfd-c929-4fa1-a5a5-60a200081227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:03.658462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a2e96dfd-c929-4fa1-a5a5-60a200081227 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:03.660828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a2e96dfd-c929-4fa1-a5a5-60a200081227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:03.666886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca3ffb-48ff-45b8-bd5c-92871ee2bfd9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:03.704219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:03.705204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:03.705645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:03.706159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:03.707088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:03.707611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:03.709185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:03.722800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 351/707] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:03.730218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:03.779288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 357/708] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:06:03.824899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e96dfd-c929-4fa1-a5a5-60a200081227 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:03.842151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85afa3c4-9f2e-4ec7-accf-eae0084bfa4b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:03.873161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:03.880894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:03.882120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:03.882650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:03.883132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:03.891417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:03.893242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:03.904765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:03.910509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:03.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:03.912270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:03.912270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:03.912270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:03.912270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:03.912270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:03.912270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:03.912270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:03.912270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:03.915316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 352/709] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:03.923490 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:04.034145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e96dfd-c929-4fa1-a5a5-60a200081227 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:04.079854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:04.080659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:04.080942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:04.081243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:04.095678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:04.097320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:04.124280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:04.125238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:04.126285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:04.126285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:04.126285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:04.126285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:04.129775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 358/710] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:04.132961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:04.185193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5fd86-6870-4b82-a86b-b2e0d17aa1dd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:04.265145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:04.266347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:04.266827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:04.267292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:04.268346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:04.268870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:04.270538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:04.284528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 353/711] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:04.294802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:04.347092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:04.362434 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafd4f6-051d-45db-b7b4-08ad3ca612b0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:04.394128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:04.453311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6658c3-4012-4263-940f-3336d14bf107 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:04.492044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:04.492778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:04.493527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:04.493957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:04.494903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=4fffb91636744fc39886bdf568317ab5) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:04.495222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:04.496769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:04.503051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b64f36-2eab-485d-af14-3f8a635cd59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:04.503375 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 354/712] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:04 2022] GET /identity/v3/role_assignments?user.id=4fffb91636744fc39886bdf568317ab5&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=5c672a1b2979444383c587a215adeb35 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:04.503748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b64f36-2eab-485d-af14-3f8a635cd59c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:04.504032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b64f36-2eab-485d-af14-3f8a635cd59c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:04.504308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b64f36-2eab-485d-af14-3f8a635cd59c None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:04.505043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b64f36-2eab-485d-af14-3f8a635cd59c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:04.505350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b64f36-2eab-485d-af14-3f8a635cd59c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:04.506800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b64f36-2eab-485d-af14-3f8a635cd59c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:04.533321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 359/713] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:04.538944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:04.679938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec60457-d3df-4478-b4e3-cbd57ba6a342 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:04.693039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:04.722309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00abd44-a347-47e0-b58b-e1d1e93cb64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:04.722905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00abd44-a347-47e0-b58b-e1d1e93cb64e None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:04.723213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00abd44-a347-47e0-b58b-e1d1e93cb64e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:04.723475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00abd44-a347-47e0-b58b-e1d1e93cb64e None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:04.724417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00abd44-a347-47e0-b58b-e1d1e93cb64e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:04.725982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00abd44-a347-47e0-b58b-e1d1e93cb64e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:04.757711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 355/714] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:06:04 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:04.787617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca280e-7cac-47b0-bfb8-bee74a837091 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:04.788194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca280e-7cac-47b0-bfb8-bee74a837091 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:04.788618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca280e-7cac-47b0-bfb8-bee74a837091 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:04.791401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 360/715] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:04.797751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b313d-b068-4505-ad0d-32604f49e524 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:04.798348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b313d-b068-4505-ad0d-32604f49e524 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:04.798799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b313d-b068-4505-ad0d-32604f49e524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:04.818979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f93b313d-b068-4505-ad0d-32604f49e524 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:04.820476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f93b313d-b068-4505-ad0d-32604f49e524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:04.907332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 356/716] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:06:04.947973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca280e-7cac-47b0-bfb8-bee74a837091 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:05.097642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca280e-7cac-47b0-bfb8-bee74a837091 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:05.144945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:05.146053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:05.146481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:05.146900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] PATH_INFO: `/v3/services/network` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:05.147735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:05.149319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:05.174369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:05.175823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:05.176869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:05.180039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 361/717] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:04 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:05.186104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b313d-b068-4505-ad0d-32604f49e524 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:05.294662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:05.301290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b313d-b068-4505-ad0d-32604f49e524 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:05.342556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93c946-7af0-4c47-932c-657e8a90c20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:05.343241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c946-7af0-4c47-932c-657e8a90c20e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:05.343526 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c946-7af0-4c47-932c-657e8a90c20e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:05.343805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93c946-7af0-4c47-932c-657e8a90c20e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:05.344533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93c946-7af0-4c47-932c-657e8a90c20e None admin] RBAC: Adding query filter params (name=network) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:05.344845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93c946-7af0-4c47-932c-657e8a90c20e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:05.346280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d93c946-7af0-4c47-932c-657e8a90c20e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:05.351774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 357/718] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:05.357431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:05.380614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:05.473522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d8b0-78a5-468a-bfa2-d457bd2407ef None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:05.515031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a5661-8609-4740-b78e-7f8f12253ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:05.515708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a5661-8609-4740-b78e-7f8f12253ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:05.515992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a5661-8609-4740-b78e-7f8f12253ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:05.516267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a5661-8609-4740-b78e-7f8f12253ca2 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:05.516982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a5661-8609-4740-b78e-7f8f12253ca2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:05.517287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a5661-8609-4740-b78e-7f8f12253ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:05.518721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a5661-8609-4740-b78e-7f8f12253ca2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:05.525807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 362/719] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:05.729851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1846841-e6d5-4d75-bc92-35743f7c86a8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:05.730428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1846841-e6d5-4d75-bc92-35743f7c86a8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:05.730428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1846841-e6d5-4d75-bc92-35743f7c86a8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:05.732094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 358/720] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:05.748279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25693f48-565a-4c5a-a73e-deb3df14218c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:05.748720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25693f48-565a-4c5a-a73e-deb3df14218c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:05.749011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25693f48-565a-4c5a-a73e-deb3df14218c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:05.773239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-25693f48-565a-4c5a-a73e-deb3df14218c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:05.774733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-25693f48-565a-4c5a-a73e-deb3df14218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:05.859565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 363/721] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:06:05.882641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1846841-e6d5-4d75-bc92-35743f7c86a8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:05.957580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da853b-8827-4307-85a1-9bd6ad039be6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:05.958250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da853b-8827-4307-85a1-9bd6ad039be6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:05.958250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da853b-8827-4307-85a1-9bd6ad039be6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:05.959742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 364/722] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:05.967088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c67b2-2ec0-42d4-a4a2-069d0a43fd5b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:05.967406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c67b2-2ec0-42d4-a4a2-069d0a43fd5b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:05.967690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c67b2-2ec0-42d4-a4a2-069d0a43fd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:06.003121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:06.006965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3b2c67b2-2ec0-42d4-a4a2-069d0a43fd5b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:06.008225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3b2c67b2-2ec0-42d4-a4a2-069d0a43fd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:06.026581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1846841-e6d5-4d75-bc92-35743f7c86a8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:06.072167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ddd48-105c-4c29-9618-904401946bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:06.073036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddd48-105c-4c29-9618-904401946bb5 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:06.073407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddd48-105c-4c29-9618-904401946bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:06.073803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ddd48-105c-4c29-9618-904401946bb5 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:06.074574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ddd48-105c-4c29-9618-904401946bb5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:06.075980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42ddd48-105c-4c29-9618-904401946bb5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:06.101190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 359/723] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:06:05 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:06.159410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 365/724] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:05 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 18 14:06:06.201499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ddd48-105c-4c29-9618-904401946bb5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:06.237373 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:06.337608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ddd48-105c-4c29-9618-904401946bb5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:06.379410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabd59e-87f0-452a-93d3-421d3a1423ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:06.380274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabd59e-87f0-452a-93d3-421d3a1423ff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:06.380566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabd59e-87f0-452a-93d3-421d3a1423ff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:06.380841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabd59e-87f0-452a-93d3-421d3a1423ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:06.393933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabd59e-87f0-452a-93d3-421d3a1423ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8eabd59e-87f0-452a-93d3-421d3a1423ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:06.401968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:06.402932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:06.403780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 360/725] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:06.409433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c67b2-2ec0-42d4-a4a2-069d0a43fd5b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:06.530674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c67b2-2ec0-42d4-a4a2-069d0a43fd5b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:06.564566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:06.571648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:06.572433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:06.572902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:06.573180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:06.573974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:06.574284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:06.575686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:06.585377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 366/726] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:06.591602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabd59e-87f0-452a-93d3-421d3a1423ff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:06.682895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:06.704221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabd59e-87f0-452a-93d3-421d3a1423ff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:06.740081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:06.741629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:06.742204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:06.742627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:06.755001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:06.756762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:06.775840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:06.777050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:06.778221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:06.778221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:06.778221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:06.778221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:06.780177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 361/727] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:06.786697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:06.957065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfe39f-cea9-45da-ba6b-f467da358d4c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:06.994708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:06.996075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:06.996522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:06.996939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:06.997845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:06.998321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:06.999959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:07.014058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 367/728] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:07.019872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:07.127783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45752-b45e-4e31-9f7c-972121d32c27 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:07.158292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:07.164107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:07.165273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:07.165944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:07.166499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:07.175103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:07.176780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:07.192930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:07.194095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:06:07.195047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:07.199055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 362/729] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:07.203762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:07.266119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:07.315346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488654fa-2eca-4635-86d0-e5e2b7de55c6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:07.352740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60ce13-f0f6-4527-a5cc-13f18b23632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:07.353854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60ce13-f0f6-4527-a5cc-13f18b23632a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:07.354371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60ce13-f0f6-4527-a5cc-13f18b23632a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:07.354876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60ce13-f0f6-4527-a5cc-13f18b23632a None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:07.355923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60ce13-f0f6-4527-a5cc-13f18b23632a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:07.356443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60ce13-f0f6-4527-a5cc-13f18b23632a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:07.358135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef60ce13-f0f6-4527-a5cc-13f18b23632a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:07.379238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 368/730] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 18 14:06:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:07.385525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:07.442072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813564-3b30-45c2-92e0-f98a36c27027 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:07.442714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813564-3b30-45c2-92e0-f98a36c27027 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:07.443387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b813564-3b30-45c2-92e0-f98a36c27027 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:07.445638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 369/731] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:07.456263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a29b78-518a-4454-a84d-75355659e950 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:07.456983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a29b78-518a-4454-a84d-75355659e950 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:07.457481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a29b78-518a-4454-a84d-75355659e950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:07.491483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-73a29b78-518a-4454-a84d-75355659e950 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:07.493765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-73a29b78-518a-4454-a84d-75355659e950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:07.505693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc805986-a42a-4df8-a70b-3b900f151ad4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:07.566761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222ff0-3d61-41bb-805f-e7ee6b3b110e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:07.567424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222ff0-3d61-41bb-805f-e7ee6b3b110e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:07.567712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222ff0-3d61-41bb-805f-e7ee6b3b110e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:07.567982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f222ff0-3d61-41bb-805f-e7ee6b3b110e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:07.628699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 370/732] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:06:07.662051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:07.678848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f222ff0-3d61-41bb-805f-e7ee6b3b110e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:07.680424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f222ff0-3d61-41bb-805f-e7ee6b3b110e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:07.680966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a29b78-518a-4454-a84d-75355659e950 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:07.765018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:07.870149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 363/733] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:07 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 18 14:06:07.876981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222ff0-3d61-41bb-805f-e7ee6b3b110e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:07.892262 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a29b78-518a-4454-a84d-75355659e950 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:07.971818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05d1d7-efc5-4647-ba15-5ebd37b9af7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:07.972560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d1d7-efc5-4647-ba15-5ebd37b9af7b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:07.972838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d1d7-efc5-4647-ba15-5ebd37b9af7b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:07.973114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d05d1d7-efc5-4647-ba15-5ebd37b9af7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:07.999879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d05d1d7-efc5-4647-ba15-5ebd37b9af7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-6d05d1d7-efc5-4647-ba15-5ebd37b9af7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:08.018643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:08.019728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:08.020693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:08.026043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 371/734] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:08.042177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05d1d7-efc5-4647-ba15-5ebd37b9af7b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:08.074046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:08.086089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f222ff0-3d61-41bb-805f-e7ee6b3b110e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:08.135639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab1d5da-e31e-45a5-9103-ddd3d2d2ae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:08.136464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d5da-e31e-45a5-9103-ddd3d2d2ae0c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.136749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d5da-e31e-45a5-9103-ddd3d2d2ae0c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.137028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab1d5da-e31e-45a5-9103-ddd3d2d2ae0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.148300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1d5da-e31e-45a5-9103-ddd3d2d2ae0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:08.150173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab1d5da-e31e-45a5-9103-ddd3d2d2ae0c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7ab1d5da-e31e-45a5-9103-ddd3d2d2ae0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:08.169579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:08.170736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:08.172246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:08.172246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:08.172246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:08.172246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:08.175882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 364/735] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:08.177049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab374fbe-e116-4b8f-91c4-84b9f1999c49 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.177346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab374fbe-e116-4b8f-91c4-84b9f1999c49 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.177606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab374fbe-e116-4b8f-91c4-84b9f1999c49 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.181133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 365/736] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:08.183263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a7481-87f8-40fa-991f-c348847feb35 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.183916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a7481-87f8-40fa-991f-c348847feb35 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.184770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a7481-87f8-40fa-991f-c348847feb35 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.185817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d05d1d7-efc5-4647-ba15-5ebd37b9af7b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:08.186821 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 366/737] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:08.187500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:08.188433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a7481-87f8-40fa-991f-c348847feb35 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:08.244857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce35dcf-c596-4db0-9620-2810d983e658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:08.245485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce35dcf-c596-4db0-9620-2810d983e658 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.245805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce35dcf-c596-4db0-9620-2810d983e658 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.246078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce35dcf-c596-4db0-9620-2810d983e658 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.246839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce35dcf-c596-4db0-9620-2810d983e658 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:08.247147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce35dcf-c596-4db0-9620-2810d983e658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:08.248599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce35dcf-c596-4db0-9620-2810d983e658 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:08.259147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 372/738] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:08.260521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c7873-b4c6-4a22-b810-32ba4b864ea8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.260814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c7873-b4c6-4a22-b810-32ba4b864ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.261097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c7873-b4c6-4a22-b810-32ba4b864ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.280699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f27c7873-b4c6-4a22-b810-32ba4b864ea8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:08.282276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f27c7873-b4c6-4a22-b810-32ba4b864ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:08.336376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a7481-87f8-40fa-991f-c348847feb35 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:08.370468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 373/739] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:06:08.372234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b83fa-3dbe-44ab-a464-9f62055effe6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.372234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b83fa-3dbe-44ab-a464-9f62055effe6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.372430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b83fa-3dbe-44ab-a464-9f62055effe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.393297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:08.394739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.395426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.396083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.397616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:08.398145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:08.400765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:08.409217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-622b83fa-3dbe-44ab-a464-9f62055effe6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:08.411151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-622b83fa-3dbe-44ab-a464-9f62055effe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:08.424645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 367/740] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:08.424645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:08.432806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:08.507255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 374/741] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 18 14:06:08.508126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b83fa-3dbe-44ab-a464-9f62055effe6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:08.540245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:08.557722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73e7b02-e9e6-4a30-84b6-afb06013436b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:08.614526 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:08.615657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.616063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.616936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.637715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:08.637715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:08.648311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b83fa-3dbe-44ab-a464-9f62055effe6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:08.658385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:08.659397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:08.660333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:08.660333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:08.660333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:08.660333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:08.660333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 368/742] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:08.660632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:08.692183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f32a8-3956-47ae-8e79-180166679d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:08.693275 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f32a8-3956-47ae-8e79-180166679d80 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.693563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f32a8-3956-47ae-8e79-180166679d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.693861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f32a8-3956-47ae-8e79-180166679d80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.705368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f32a8-3956-47ae-8e79-180166679d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d11f32a8-3956-47ae-8e79-180166679d80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:08.713624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:08.714740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:06:08.715790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:08.719723 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 375/743] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:06:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:08.721043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f32a8-3956-47ae-8e79-180166679d80 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:08.730642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:08.794023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4edce-001a-47ee-9dc8-f7019e45c61e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:08.829916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:08.847425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:08.848405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.848836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.849121 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.849254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f32a8-3956-47ae-8e79-180166679d80 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:08.861885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:08.863372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:08.891008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:08.892540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:08.893416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:08.893416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:08.893416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:08.893416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:08.893416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:08.893416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:08.893416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:08.893416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:08.893896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3e593-969a-424c-ae82-11644a0217a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:08.893896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3e593-969a-424c-ae82-11644a0217a2 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:08.894058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 369/744] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:08.894129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3e593-969a-424c-ae82-11644a0217a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:08.894129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3e593-969a-424c-ae82-11644a0217a2 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:08.894261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:08.894329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3e593-969a-424c-ae82-11644a0217a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:08.895032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f3e593-969a-424c-ae82-11644a0217a2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:08.916550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 376/745] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:06:08 2022] POST /identity/v3/services => generated 240 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:08.917519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3e593-969a-424c-ae82-11644a0217a2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:09.058185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79ca05-831e-4a14-9c4c-7576a0fbf3ca None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:09.082791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:09.115712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f3e593-969a-424c-ae82-11644a0217a2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:09.136249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cfbaf-d470-4ac7-99a1-de4b8e52a0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:09.137226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cfbaf-d470-4ac7-99a1-de4b8e52a0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.137533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cfbaf-d470-4ac7-99a1-de4b8e52a0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.137854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cfbaf-d470-4ac7-99a1-de4b8e52a0e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.138669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cfbaf-d470-4ac7-99a1-de4b8e52a0e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:09.139021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cfbaf-d470-4ac7-99a1-de4b8e52a0e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:09.140424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cfbaf-d470-4ac7-99a1-de4b8e52a0e2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:09.156224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 370/746] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:09.158460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15ba8d-1409-493c-bd67-c6869493bdcb None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.158785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15ba8d-1409-493c-bd67-c6869493bdcb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.159035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15ba8d-1409-493c-bd67-c6869493bdcb None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.160810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 371/747] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:09.162833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15ba8d-1409-493c-bd67-c6869493bdcb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:09.167231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:09.169169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.169730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.170149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.171073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:09.171380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:09.172815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:09.184837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 377/748] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:06:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:09.188326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:09.196430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:09.307568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15ba8d-1409-493c-bd67-c6869493bdcb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:09.346385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647ce0a-b839-46fc-803f-2fc7477eea6e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:09.346984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e8155-caa6-4f0c-8a34-51ebf2fe249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:09.348219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e8155-caa6-4f0c-8a34-51ebf2fe249a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.348829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e8155-caa6-4f0c-8a34-51ebf2fe249a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.349301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e8155-caa6-4f0c-8a34-51ebf2fe249a None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.350527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e8155-caa6-4f0c-8a34-51ebf2fe249a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:09.350874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e8155-caa6-4f0c-8a34-51ebf2fe249a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:09.352518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e8155-caa6-4f0c-8a34-51ebf2fe249a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:09.373793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 372/749] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:09.375576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ba295-ee3b-42fd-915f-aac99d9aaa59 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.376018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ba295-ee3b-42fd-915f-aac99d9aaa59 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.376302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ba295-ee3b-42fd-915f-aac99d9aaa59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.398749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:09.399929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bd0ba295-ee3b-42fd-915f-aac99d9aaa59 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:09.400113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.400547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.400977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.401920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bd0ba295-ee3b-42fd-915f-aac99d9aaa59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:09.410785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:09.413587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:09.433809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:09.434951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:09.435997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:09.438010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 378/750] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:06:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:09.439145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:09.446337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:09.509691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 373/751] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:09.509691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ba295-ee3b-42fd-915f-aac99d9aaa59 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:09.551213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:09.609008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8afc33-2703-4752-ab03-133500a0d68b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:09.659897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:09.660713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.660994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.661270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.665059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ba295-ee3b-42fd-915f-aac99d9aaa59 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:09.681359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:09.682920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:09.707528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:09.708521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:09.709442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:09.709442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:09.709442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:09.709442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:09.709442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 379/752] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:09.710658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:09.720915 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:09.721973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.722431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.722878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.723902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=203e654f95bf45e683b0ccc4ba687bfc) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:09.724473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:09.726229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:09.734562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 374/753] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:09 2022] GET /identity/v3/role_assignments?user.id=203e654f95bf45e683b0ccc4ba687bfc&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=5c672a1b2979444383c587a215adeb35 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:09.735994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:09.796787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:09.891089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4c34b-77cb-4ae5-848c-c0c5541d7d00 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:09.921081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb4297-d05d-4fdf-a0c6-27b0131177e9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:09.941150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:09.942379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.942809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.943256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.944071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:09.945640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:09.948609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:09.960186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:09.962328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:09.964974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 375/754] 10.222.0.71 () {56 vars in 1129 bytes} [Mon Jul 18 14:06:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:09.967277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:09.982241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:09.982241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:09.982241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:09.982775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:09.983204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:09.983520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:09.985608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.017690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 380/755] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:06:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:10.017690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:10.130001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:10.138688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a72de-0920-4374-b723-1c7aa3f68bb8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:10.140853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760e745-ade4-48f7-a72d-8f79eedabe83 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:10.179236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:10.179995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:10.180398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:10.180772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:10.181626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:10.182076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:10.183576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.195926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:10.196982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:10.197438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:10.197880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:10.202911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 376/756] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:10.202911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:10.203177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:10.203177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:10.204206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.214255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 381/757] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:10.215691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:10.273921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:10.367855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5581920-40d3-45de-ad70-c43c5b6037d0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:10.373218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140f891-f760-4b0a-a8b8-32651b4ab2c5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:10.410042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:10.411133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:10.411598 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:10.411964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:10.421154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:10.422021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:10.423009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.423129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:10.423531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:10.423965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:10.436741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:10.438456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:10.440834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:10.442038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:10.443215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:10.449888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 382/758] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:10.449888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:10.473760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:10.474941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:10.475958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:10.475958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:10.475958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:10.475958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:10.483230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 377/759] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:10.484662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:10.497888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:10.590323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:10.597863 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f54ce06-05f0-4c0b-b18e-f6f544b2a2ba None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:10.650343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a692848-9cf5-4461-8cf2-b012562ac6e0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:10.672184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:10.673158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:10.673600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:10.674048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:10.674985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:10.676039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:10.677946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.690960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 383/760] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:10.690960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:10.699847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:10.701497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:10.702041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:10.702544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:10.704125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:10.705061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:10.707117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.738785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 378/761] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:10.741413 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:10.815649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:10.832031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2fe17-88d7-4f2a-a856-4f0303b88898 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:10.864707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79cf05-4931-4a04-8b93-7d481a31cb1f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:10.879538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:10.880391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:10.880505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:10.880769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:10.881488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:10.881837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:10.883211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.902767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 384/762] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:10.903755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:10.914505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:10.916443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:10.916945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:10.917842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:10.919159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] RBAC: Adding query filter params (interface=public, service_id=52ac79fac104481dac8cbc8b92ed9158, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:10.919693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:10.922108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:10.928009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:10.941684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 379/763] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/endpoints?service_id=52ac79fac104481dac8cbc8b92ed9158&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:10.942919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:11.020294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babfc7f-b7c7-46ef-9e71-a242350df217 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:11.057105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d64c9-d746-4994-8cef-9962e2b02212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:11.057792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d64c9-d746-4994-8cef-9962e2b02212 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.058071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d64c9-d746-4994-8cef-9962e2b02212 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.058348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d64c9-d746-4994-8cef-9962e2b02212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.078662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae0ac-cbd1-46ab-be46-d7e38c711779 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:11.113737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:11.126958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:11.128136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.128875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.129374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.133451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d64c9-d746-4994-8cef-9962e2b02212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:11.135364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d64c9-d746-4994-8cef-9962e2b02212 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:11.142203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:11.144373 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:11.157460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:11.163412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:11.164387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:11.165319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:11.165319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:11.165319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:11.165319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:11.165319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:11.165319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:11.165319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:11.165319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:11.170092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 380/764] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:11.171104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:11.253995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 385/765] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:11.255190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d64c9-d746-4994-8cef-9962e2b02212 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:11.290913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1596a-8e51-43b6-bda1-683dc7b3d3ac None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:11.328253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc6a99-8af5-4734-9dca-a3197ed5158f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:11.328933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc6a99-8af5-4734-9dca-a3197ed5158f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.329219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc6a99-8af5-4734-9dca-a3197ed5158f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.329500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc6a99-8af5-4734-9dca-a3197ed5158f None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.330226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc6a99-8af5-4734-9dca-a3197ed5158f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:11.331622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc6a99-8af5-4734-9dca-a3197ed5158f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:11.338387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 381/766] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:06:11 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:11.339789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff76df-3a24-4368-9d22-37a3dd017db8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.340082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff76df-3a24-4368-9d22-37a3dd017db8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.340357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff76df-3a24-4368-9d22-37a3dd017db8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.341822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 382/767] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:11.346830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff76df-3a24-4368-9d22-37a3dd017db8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:11.377873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d64c9-d746-4994-8cef-9962e2b02212 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:11.402100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:11.414548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:11.435104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f80d3-128a-49e1-b4f5-afb0a80e30b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:11.435792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f80d3-128a-49e1-b4f5-afb0a80e30b5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.436068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f80d3-128a-49e1-b4f5-afb0a80e30b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.436333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f80d3-128a-49e1-b4f5-afb0a80e30b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.437146 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f80d3-128a-49e1-b4f5-afb0a80e30b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:11.437534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f80d3-128a-49e1-b4f5-afb0a80e30b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:11.438954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f80d3-128a-49e1-b4f5-afb0a80e30b5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:11.459010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 386/768] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:11.459010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911324-fac7-458f-b8f0-5497c06d30e1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.459010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911324-fac7-458f-b8f0-5497c06d30e1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.459010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911324-fac7-458f-b8f0-5497c06d30e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.483557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-78911324-fac7-458f-b8f0-5497c06d30e1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:11.485516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-78911324-fac7-458f-b8f0-5497c06d30e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:11.499124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff76df-3a24-4368-9d22-37a3dd017db8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:11.543373 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:11.545507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.546015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.546470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.558568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:11.560785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:11.581770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:11.582762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:11.583654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:11.583654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:11.583654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:11.583654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:11.583654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 383/769] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:11.584408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:11.590561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 387/770] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:11.591728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78911324-fac7-458f-b8f0-5497c06d30e1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:11.704010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd3ae0-56a6-4fa9-bec1-b8620934063b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:11.706780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:11.735986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78911324-fac7-458f-b8f0-5497c06d30e1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:11.745949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77762159-d2f8-4109-a56a-0a76f6e7f6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:11.746751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762159-d2f8-4109-a56a-0a76f6e7f6ee None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.747137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762159-d2f8-4109-a56a-0a76f6e7f6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.747536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762159-d2f8-4109-a56a-0a76f6e7f6ee None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/4fffb91636744fc39886bdf568317ab5/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.750010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:11.750386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77762159-d2f8-4109-a56a-0a76f6e7f6ee None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=4fffb91636744fc39886bdf568317ab5, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:11.751896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77762159-d2f8-4109-a56a-0a76f6e7f6ee None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:11.773827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 384/771] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:06:11 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/4fffb91636744fc39886bdf568317ab5/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:06:11.773827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77762159-d2f8-4109-a56a-0a76f6e7f6ee None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:11.800788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:11.802616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:11.804633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:11.804938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:11.807596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:11.807913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:11.811572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:11.841511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 388/772] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:11.847256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:11.964101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bad5d8-bb7c-49b0-8b9f-d7a718cc455d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:11.977449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77762159-d2f8-4109-a56a-0a76f6e7f6ee None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.010374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85f382-d186-457e-944d-718b0fd69774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.011996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f382-d186-457e-944d-718b0fd69774 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.012608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f382-d186-457e-944d-718b0fd69774 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.013169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f382-d186-457e-944d-718b0fd69774 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.020343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d58429-304c-400e-948d-87174dddeb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.021162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d58429-304c-400e-948d-87174dddeb62 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.021453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d58429-304c-400e-948d-87174dddeb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.021809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d58429-304c-400e-948d-87174dddeb62 None admin] PATH_INFO: `/v3/services/network` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.022481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d58429-304c-400e-948d-87174dddeb62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.022589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85f382-d186-457e-944d-718b0fd69774 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.023867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d58429-304c-400e-948d-87174dddeb62 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.024606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85f382-d186-457e-944d-718b0fd69774 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-25d58429-304c-400e-948d-87174dddeb62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:12.036243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:12.037283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:12.038219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 385/773] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:11 2022] GET /identity/v3/services/network => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:12.039088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4e3b1-537d-4c94-ab75-67ae579b1d6a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.039391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4e3b1-537d-4c94-ab75-67ae579b1d6a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.039605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4e3b1-537d-4c94-ab75-67ae579b1d6a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.041143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 386/774] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-bf85f382-d186-457e-944d-718b0fd69774 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:12.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:12.047084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:12.048224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:12.048224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:12.048224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:12.048224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:12.048224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:12.048224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:12.048224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:12.048224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:12.048764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4e3b1-537d-4c94-ab75-67ae579b1d6a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.050523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 389/775] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:12.053419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabf358-d660-46e8-9570-a111ffa027b7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.053901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabf358-d660-46e8-9570-a111ffa027b7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.054322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabf358-d660-46e8-9570-a111ffa027b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.076962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8aabf358-d660-46e8-9570-a111ffa027b7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:12.079101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8aabf358-d660-46e8-9570-a111ffa027b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:12.080179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:12.119127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:12.165911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4e3b1-537d-4c94-ab75-67ae579b1d6a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.173703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 390/776] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:12.175654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabf358-d660-46e8-9570-a111ffa027b7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.229773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.231007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.231566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.231852 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.232802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] RBAC: Adding query filter params (name=network) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:12.233102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.234824 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.241864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 387/777] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:12.243134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.292039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabf358-d660-46e8-9570-a111ffa027b7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.343625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.344216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.344499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.344776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.346382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:12.347046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.348782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.349628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:12.362819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 391/778] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:12.363036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a421a9-78cd-4d8a-84fb-017367a05904 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.363666 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.386473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:12.403003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e24cf-7145-46cb-a933-dfef72d945e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.403925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e24cf-7145-46cb-a933-dfef72d945e6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.404281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e24cf-7145-46cb-a933-dfef72d945e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.404629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e24cf-7145-46cb-a933-dfef72d945e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.416825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e24cf-7145-46cb-a933-dfef72d945e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-428e24cf-7145-46cb-a933-dfef72d945e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:12.425316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:12.426397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:12.427250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:12.432011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 388/779] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:12.433076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e24cf-7145-46cb-a933-dfef72d945e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.468812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558988-8932-4eeb-bfda-6f5eb460a7ca None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.505309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.506036 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.506305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.506580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.507387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] RBAC: Adding query filter params (type=network) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:12.507701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.509498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.515052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 392/780] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:12.515970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.541612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e24cf-7145-46cb-a933-dfef72d945e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.578225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.578894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.579243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.579622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.580554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=1ecd34d877d4476d86e0e8b4ae9c56c9) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:12.580877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.582464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.588865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 389/781] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/role_assignments?user.id=1ecd34d877d4476d86e0e8b4ae9c56c9&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:12.589788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.610664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:12.611552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:12.636992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cff032-5cfe-4dd8-9419-e3551f9dfa83 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.677761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.678543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.678855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.679219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.680001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:12.680347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.681914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.692473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 393/782] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:12.698437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.738105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0a1de-2bf2-48d6-8bbf-fe6a5b33e097 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.778095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.779069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.780312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.780591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.781416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] RBAC: Adding query filter params (interface=public, service_id=af7b46227c30411285abca21f2af320b, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:12.781759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.783181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.791265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 390/783] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/endpoints?service_id=af7b46227c30411285abca21f2af320b&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:12.796613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.828017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fff14-5161-4264-a1da-645937128ee5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:12.869473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:12.870148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:12.870430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:12.870705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:12.895101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:12.899573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:12.914011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:12.929211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:12.931393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:12.932983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:12.932983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:12.932983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:12.932983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:12.932983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 394/784] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:12.939729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:12.947010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:12.973618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb8143-7a78-4ee5-b1f8-e31b62713b38 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:13.039074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ae251-7f52-4989-bb89-4f84dee74432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:13.039989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ae251-7f52-4989-bb89-4f84dee74432 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:13.040369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ae251-7f52-4989-bb89-4f84dee74432 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:13.040832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ae251-7f52-4989-bb89-4f84dee74432 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:13.041533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ae251-7f52-4989-bb89-4f84dee74432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:13.042977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ae251-7f52-4989-bb89-4f84dee74432 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:13.055402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 391/785] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/services => generated 772 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:13.136142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3cf24-167d-498b-a2b6-5c7ec69ca8b2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:13.191225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:13.192082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:13.192273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:13.192561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:13.193222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:13.193535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:13.195144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:13.215088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 395/786] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:13.220460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ae251-7f52-4989-bb89-4f84dee74432 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:13.376353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ae251-7f52-4989-bb89-4f84dee74432 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:13.424809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:13.425597 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:13.425936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:13.426214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:13.437725 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:13.439214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:13.458216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:13.460498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:13.462221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 392/787] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:13.466834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:13.532271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:13.584509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829aa96-56ad-4759-8cfb-6940e71f3dad None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:13.626329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205becff-8962-4445-af64-8ed765486716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:13.627348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205becff-8962-4445-af64-8ed765486716 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:13.627720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205becff-8962-4445-af64-8ed765486716 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:13.627999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205becff-8962-4445-af64-8ed765486716 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:13.629696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205becff-8962-4445-af64-8ed765486716 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:13.630033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205becff-8962-4445-af64-8ed765486716 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:13.631626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205becff-8962-4445-af64-8ed765486716 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:13.663442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 396/788] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 18 14:06:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:13.674721 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:13.678905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:13.790866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c62c2f-4e7f-4329-b675-e9b8ae19f3a6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:13.791266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c62c2f-4e7f-4329-b675-e9b8ae19f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:13.791617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c62c2f-4e7f-4329-b675-e9b8ae19f3a6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:13.797608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 397/789] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:13.798677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c7d97-dd9e-4dcb-b55b-185c735578a1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:13.803236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657a92-32d3-4d0d-a045-de575e92a17c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:13.803519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657a92-32d3-4d0d-a045-de575e92a17c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:13.803789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a657a92-32d3-4d0d-a045-de575e92a17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:13.833463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4a657a92-32d3-4d0d-a045-de575e92a17c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:13.834008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:13.834707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4a657a92-32d3-4d0d-a045-de575e92a17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:13.834917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:13.835107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:13.835382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:13.847045 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:13.848521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:13.869084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:13.870088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:13.870972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:13.870972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:13.870972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:13.870972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:13.870972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 393/790] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:13.876308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:13.948570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:13.982180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4425a-f130-4164-96a0-0b9f3cc395b8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.013076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 398/791] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:14.051282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a657a92-32d3-4d0d-a045-de575e92a17c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.052322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:14.053016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.053503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.053827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.055700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:14.056276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:14.057755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:14.078340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 394/792] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:14.083087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.134743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:14.189296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a657a92-32d3-4d0d-a045-de575e92a17c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.237414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a82f9-51a7-416c-ad07-6d119b2dda30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:14.238223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a82f9-51a7-416c-ad07-6d119b2dda30 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.238467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a82f9-51a7-416c-ad07-6d119b2dda30 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.238735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a82f9-51a7-416c-ad07-6d119b2dda30 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.239924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a82f9-51a7-416c-ad07-6d119b2dda30 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:14.241867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5a82f9-51a7-416c-ad07-6d119b2dda30 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:14.252055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370622d3-fc21-4fab-9ef8-94b0f06cee3a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-6a5a82f9-51a7-416c-ad07-6d119b2dda30 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:14.258227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:14.259244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 18 14:06:14.260085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:14.260866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 399/793] 10.222.0.71 () {56 vars in 1129 bytes} [Mon Jul 18 14:06:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:14.262067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4113-e8e5-424f-9e31-d01697d82b8d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.262350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4113-e8e5-424f-9e31-d01697d82b8d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.265678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e4113-e8e5-424f-9e31-d01697d82b8d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.268686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 400/794] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:14.269637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e4113-e8e5-424f-9e31-d01697d82b8d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.277707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:14.297724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b7e27-8947-441d-83d7-3632bf9c01c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:14.298744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b7e27-8947-441d-83d7-3632bf9c01c7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.299136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b7e27-8947-441d-83d7-3632bf9c01c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.299485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b7e27-8947-441d-83d7-3632bf9c01c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.306927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b7e27-8947-441d-83d7-3632bf9c01c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:14.308696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b7e27-8947-441d-83d7-3632bf9c01c7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-689b7e27-8947-441d-83d7-3632bf9c01c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:14.324220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:14.326331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:14.329075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 395/795] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:14.330088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dce903-e182-4fee-a6c9-8295b17145c8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.330088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dce903-e182-4fee-a6c9-8295b17145c8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.330347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dce903-e182-4fee-a6c9-8295b17145c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.350260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-99dce903-e182-4fee-a6c9-8295b17145c8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:14.351686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-99dce903-e182-4fee-a6c9-8295b17145c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:14.397727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e4113-e8e5-424f-9e31-d01697d82b8d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.411513 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:14.429841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 396/796] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:14.431418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dce903-e182-4fee-a6c9-8295b17145c8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.454978 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:14.456001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.456464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.456916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.458177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:14.458491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:14.459961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:14.468464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 401/797] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 18 14:06:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:14.469622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.564883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dce903-e182-4fee-a6c9-8295b17145c8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.574950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:14.597544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70a6fb-2676-4d51-9db1-1b2c1c2bcd4d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.607275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:14.607961 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.608248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.608530 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.609324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:14.609686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:14.611279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:14.625535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 397/798] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:14.626447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.640546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea5189-3b22-4c4e-a911-ed53ea1ec909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:14.641505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea5189-3b22-4c4e-a911-ed53ea1ec909 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.642526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea5189-3b22-4c4e-a911-ed53ea1ec909 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.642526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea5189-3b22-4c4e-a911-ed53ea1ec909 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.655104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea5189-3b22-4c4e-a911-ed53ea1ec909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e4ea5189-3b22-4c4e-a911-ed53ea1ec909 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:14.663370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:14.664454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:14.665400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:14.669929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 402/799] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:14.670986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea5189-3b22-4c4e-a911-ed53ea1ec909 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.709456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:14.755230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbead3-dd2b-40a9-912a-da048d283c73 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.781574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea5189-3b22-4c4e-a911-ed53ea1ec909 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.793984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:14.795003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.795470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.795922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.796908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:14.797421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:14.799064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:14.809707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 398/800] 10.222.0.71 () {56 vars in 1124 bytes} [Mon Jul 18 14:06:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:14.810989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.819294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b057c9b-3fa0-4141-a81f-fde20fc534b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:14.820197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b057c9b-3fa0-4141-a81f-fde20fc534b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:14.820539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b057c9b-3fa0-4141-a81f-fde20fc534b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:14.820928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b057c9b-3fa0-4141-a81f-fde20fc534b1 None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/203e654f95bf45e683b0ccc4ba687bfc/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:14.823425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b057c9b-3fa0-4141-a81f-fde20fc534b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=203e654f95bf45e683b0ccc4ba687bfc, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:14.825686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b057c9b-3fa0-4141-a81f-fde20fc534b1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:14.845274 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 403/801] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:06:14 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/203e654f95bf45e683b0ccc4ba687bfc/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:06:14.846402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b057c9b-3fa0-4141-a81f-fde20fc534b1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:14.848296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:14.938282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870898-6751-47ef-9fdb-75367a28da80 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:14.981690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:14.999440 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b057c9b-3fa0-4141-a81f-fde20fc534b1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:15.026525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:15.027399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.027811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.028196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.029313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:15.029501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:15.031235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:15.049252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 399/802] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:15.051464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:15.071043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a772a-19ea-4b3a-b9b6-dfd79a7729ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:15.071654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a772a-19ea-4b3a-b9b6-dfd79a7729ff None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.071942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a772a-19ea-4b3a-b9b6-dfd79a7729ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.072214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a772a-19ea-4b3a-b9b6-dfd79a7729ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.072975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a772a-19ea-4b3a-b9b6-dfd79a7729ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:15.074566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a772a-19ea-4b3a-b9b6-dfd79a7729ff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:15.114238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 404/803] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:06:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:15.170400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:15.180058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909ae0e-fc90-4130-b306-8f26cc5c8555 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:15.232977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbabca0-e10e-4d0c-9c8c-078da55f7709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:15.234229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbabca0-e10e-4d0c-9c8c-078da55f7709 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.234678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbabca0-e10e-4d0c-9c8c-078da55f7709 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.235108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbabca0-e10e-4d0c-9c8c-078da55f7709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.248776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbabca0-e10e-4d0c-9c8c-078da55f7709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:15.250687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbabca0-e10e-4d0c-9c8c-078da55f7709 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4dbabca0-e10e-4d0c-9c8c-078da55f7709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:15.270057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:15.271231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:15.272337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:15.272337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:15.272337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:15.272337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:15.279071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 400/804] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:15.280762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a772a-19ea-4b3a-b9b6-dfd79a7729ff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:15.310722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2111f-a852-479e-85c1-126a47b0fc75 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.311312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2111f-a852-479e-85c1-126a47b0fc75 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.311753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2111f-a852-479e-85c1-126a47b0fc75 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.313312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 401/805] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:15.319428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1473cee-dc61-4a65-aee7-149acdf37d6f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.319950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1473cee-dc61-4a65-aee7-149acdf37d6f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.320368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1473cee-dc61-4a65-aee7-149acdf37d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.351860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d1473cee-dc61-4a65-aee7-149acdf37d6f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:15.359230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d1473cee-dc61-4a65-aee7-149acdf37d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:15.449241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a772a-19ea-4b3a-b9b6-dfd79a7729ff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:15.451404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 402/806] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:15.475412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1473cee-dc61-4a65-aee7-149acdf37d6f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:15.490370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:15.491421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.492200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.492622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.494111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:15.494595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:15.496626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:15.517841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 405/807] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:15.519564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:15.537841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:15.578030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:15.600034 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1473cee-dc61-4a65-aee7-149acdf37d6f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:15.646024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc56a28-72a7-4c91-baa7-2895f46017c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:15.646869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc56a28-72a7-4c91-baa7-2895f46017c8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.647812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc56a28-72a7-4c91-baa7-2895f46017c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.648306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc56a28-72a7-4c91-baa7-2895f46017c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.653976 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e6487-1bc7-480b-adeb-8566a8fe6f18 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:15.665119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc56a28-72a7-4c91-baa7-2895f46017c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7dc56a28-72a7-4c91-baa7-2895f46017c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:15.681217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:15.683014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:06:15.684517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:15.688035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 403/808] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:06:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:15.689929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa794a2-048b-4c1f-af82-5c9dc37fb202 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.690422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa794a2-048b-4c1f-af82-5c9dc37fb202 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.690894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa794a2-048b-4c1f-af82-5c9dc37fb202 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.692914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 404/809] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:15.693042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6982868-28fd-4f79-acf3-eaa8ea51d5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:15.693781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6982868-28fd-4f79-acf3-eaa8ea51d5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.694079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6982868-28fd-4f79-acf3-eaa8ea51d5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.694339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6982868-28fd-4f79-acf3-eaa8ea51d5f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.694633 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa794a2-048b-4c1f-af82-5c9dc37fb202 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:15.706876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6982868-28fd-4f79-acf3-eaa8ea51d5f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:15.708579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6982868-28fd-4f79-acf3-eaa8ea51d5f4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a6982868-28fd-4f79-acf3-eaa8ea51d5f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:15.724137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:15.725143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:15.726523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:15.730286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 406/810] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:06:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:15.731649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daedb9-9ebc-4e4c-9352-4d07ddfa682d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.731944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daedb9-9ebc-4e4c-9352-4d07ddfa682d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.732239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81daedb9-9ebc-4e4c-9352-4d07ddfa682d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.750629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-81daedb9-9ebc-4e4c-9352-4d07ddfa682d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:15.751846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-81daedb9-9ebc-4e4c-9352-4d07ddfa682d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:15.807395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa794a2-048b-4c1f-af82-5c9dc37fb202 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:15.823815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:15.834078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 407/811] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:15.835169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daedb9-9ebc-4e4c-9352-4d07ddfa682d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:15.840936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:15.847807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:15.848798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:15.849300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:15.849827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:15.850791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:15.851304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:15.853115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:15.862729 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 405/812] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:06:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:15.869296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:15.969915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81daedb9-9ebc-4e4c-9352-4d07ddfa682d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:15.980060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043475af-fa18-4940-9b07-af67b764b96c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.008208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.008972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.009233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.009566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.010408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:16.010726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.012401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.017054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.018182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.018661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.019148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.031161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.033036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.036700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 408/813] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:06:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:16.037623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:16.052547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:16.053474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:16.054712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:16.054712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:16.054712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:16.054712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:16.057087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 406/814] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:16.058292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.108508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:16.146041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:16.162031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa64b4-8724-490b-a173-b99932a48d42 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.167465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c5fbd-86ce-45e9-bf9b-acebacee1868 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.206968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c7755-e988-4694-b890-2350bb4f4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.208049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c7755-e988-4694-b890-2350bb4f4383 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.208336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c7755-e988-4694-b890-2350bb4f4383 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.208615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c7755-e988-4694-b890-2350bb4f4383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.212840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.214873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.215161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.215442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] PATH_INFO: `/v3/services/network` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.216110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.217528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:16.233586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:16.234659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:16.236122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 409/815] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:16.236936 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.304825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c7755-e988-4694-b890-2350bb4f4383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.306881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c7755-e988-4694-b890-2350bb4f4383 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.349465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3e6ca-5390-4515-9a13-cc50c8397d67 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.364226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:16.382808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:16.397532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.399063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.399594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.400043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.401303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:16.401811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.403983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.424399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 410/816] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:16.425364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.434877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 407/817] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:16.436088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c7755-e988-4694-b890-2350bb4f4383 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.541071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c7755-e988-4694-b890-2350bb4f4383 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.561028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f2282-edd7-49a1-a514-74d9ec0c8c37 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.579394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.580421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.580810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.581191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.590813 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.592249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.607129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.608098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.608550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.608959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:16.609626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:16.610743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:16.611878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:16.612747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] RBAC: Adding query filter params (name=network) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:16.613204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.614756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.619154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 408/818] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:16.620200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.621432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 411/819] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:16.622527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.624469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:16.679639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:16.737839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1fd17-5c3d-469d-aee4-bebfcea6a191 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.741828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce28e4c-06e1-491d-8330-7a766a42dd9d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.777988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.778922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.779223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.779642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.780019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.780246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.780488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.780862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.781607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:16.782146 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.783817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.792792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.795071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:16.808706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 412/820] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:16.809847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:16.813365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:16.814331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:16.815241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:16.815241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:16.815241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:16.815241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:16.821236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 409/821] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:16.821236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:16.896068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:16.939171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d247d-7a06-4ae9-8ec4-820d476ab34a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.942469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:16.961207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a627b-de87-4d74-be78-2baffd0d5003 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:16.988831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:16.989879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:16.990568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:16.990982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:16.992468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] RBAC: Adding query filter params (type=network) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:16.993102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:16.995666 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.000261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f017b41-ea55-49ef-9d8f-71fe1a15592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.001165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017b41-ea55-49ef-9d8f-71fe1a15592f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.001570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017b41-ea55-49ef-9d8f-71fe1a15592f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.001967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f017b41-ea55-49ef-9d8f-71fe1a15592f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.003145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 413/822] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:17.004237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.014161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f017b41-ea55-49ef-9d8f-71fe1a15592f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:17.015680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f017b41-ea55-49ef-9d8f-71fe1a15592f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-7f017b41-ea55-49ef-9d8f-71fe1a15592f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:17.034880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:17.035847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:17.036718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:17.036718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:17.036718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:17.036718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:17.036718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 410/823] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:17.037556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4afdc8-6300-4daf-8465-bed22b5cbae6 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.037757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4afdc8-6300-4daf-8465-bed22b5cbae6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.037926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4afdc8-6300-4daf-8465-bed22b5cbae6 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.044167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 411/824] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:17.044167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4afdc8-6300-4daf-8465-bed22b5cbae6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.142254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f7b98-f302-4d3e-bb31-f9e4e4779385 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:17.156290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4afdc8-6300-4daf-8465-bed22b5cbae6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:17.167306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:17.179670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.180617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.181308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.181589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.182351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:17.182677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:17.184054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.195954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 414/825] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:17.197503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.201755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b306d36b-148e-4cfd-a7bd-a53ed840f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.202519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306d36b-148e-4cfd-a7bd-a53ed840f2e6 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.202804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306d36b-148e-4cfd-a7bd-a53ed840f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.203078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b306d36b-148e-4cfd-a7bd-a53ed840f2e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.204000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306d36b-148e-4cfd-a7bd-a53ed840f2e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:17.205580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b306d36b-148e-4cfd-a7bd-a53ed840f2e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.230651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 412/826] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:06:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:17.232069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f938-0e8e-4fd6-b050-716e4d61f345 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.232363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f938-0e8e-4fd6-b050-716e4d61f345 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.232646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0f938-0e8e-4fd6-b050-716e4d61f345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.246668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:17.253938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ead0f938-0e8e-4fd6-b050-716e4d61f345 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:17.255339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ead0f938-0e8e-4fd6-b050-716e4d61f345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:17.346469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 413/827] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:17.347508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0f938-0e8e-4fd6-b050-716e4d61f345 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.355142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce2cba-2146-4855-a3f7-6e7caf5040c7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:17.406235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.407291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.408427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.409001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.410299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:17.410833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:17.413430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.431572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 415/828] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:17.437783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.455468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:17.470994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0f938-0e8e-4fd6-b050-716e4d61f345 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:17.524526 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.524526 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.524841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.524914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.536116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:17.538054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.550637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:17.555032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:17.556255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 414/829] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:17.557442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.576778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ba513-f16f-4933-a0c5-2e3ef0cb63e8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:17.627654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c976-fedd-4e97-b21e-f4e95de70564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.628372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c976-fedd-4e97-b21e-f4e95de70564 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.628654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c976-fedd-4e97-b21e-f4e95de70564 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.628926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c976-fedd-4e97-b21e-f4e95de70564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.640944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281c976-fedd-4e97-b21e-f4e95de70564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-6281c976-fedd-4e97-b21e-f4e95de70564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:17.648700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:17.650131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:17.651096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:17.651821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 416/830] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:17.652845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12427ca-8f61-4b94-8bd9-852423aded08 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.653137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12427ca-8f61-4b94-8bd9-852423aded08 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.653416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12427ca-8f61-4b94-8bd9-852423aded08 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.659732 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 417/831] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:17.660595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12427ca-8f61-4b94-8bd9-852423aded08 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.692699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdcec3-a1a2-4b10-afa3-6c0794ffaee0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:17.728930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.729965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.730116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.730400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.737615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:17.739111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:17.754318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:17.755554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 415/832] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:17.756653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.769773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:17.784721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12427ca-8f61-4b94-8bd9-852423aded08 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:17.818961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032b9c-516a-479e-a46c-6b75118a6542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.819634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032b9c-516a-479e-a46c-6b75118a6542 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.819917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032b9c-516a-479e-a46c-6b75118a6542 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.820190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032b9c-516a-479e-a46c-6b75118a6542 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.821406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032b9c-516a-479e-a46c-6b75118a6542 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:17.821763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032b9c-516a-479e-a46c-6b75118a6542 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:17.823212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032b9c-516a-479e-a46c-6b75118a6542 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.835113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 418/833] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:17.836375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483d90c-7389-457a-8bdd-465e8fd65d4e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.836759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483d90c-7389-457a-8bdd-465e8fd65d4e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.837049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483d90c-7389-457a-8bdd-465e8fd65d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.853180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2483d90c-7389-457a-8bdd-465e8fd65d4e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:17.854423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2483d90c-7389-457a-8bdd-465e8fd65d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:17.856453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:17.865892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252e392-9e70-4fb7-92e5-8068fc4a3d5c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:17.904499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:17.905177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:17.905464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:17.905776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:17.906533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:17.906844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:17.908308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:17.916685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 416/834] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:17.917534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.927822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 419/835] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:17.928669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483d90c-7389-457a-8bdd-465e8fd65d4e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:17.999982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:18.031504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a69a4-71bb-45f3-bf64-798c533f7128 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.036273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483d90c-7389-457a-8bdd-465e8fd65d4e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.068507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.069188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.069474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.069782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.070578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:18.071020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.071360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.072127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.072405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.072673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.073095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.073513 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=4fffb91636744fc39886bdf568317ab5) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:18.073861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.075282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.081442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 420/836] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/role_assignments?user.id=4fffb91636744fc39886bdf568317ab5&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=5c672a1b2979444383c587a215adeb35 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:18.082289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:18.085239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:18.085698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 417/837] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:18.091307 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:18.216247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677957d2-6e7c-4174-be08-3290656ddf5a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.217967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65848744-d5c7-4711-ae38-c881fb1bdbf1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.259421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:18.260062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.261497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.261497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.261497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.265376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.266223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.266951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.266951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.267408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.268795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:18.282850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:18.284037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:18.284962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 418/838] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:18.285999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:18.393711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.395215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:18.413947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:18.415477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:18.416384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:18.416384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:18.416384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:18.416384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:18.416384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 421/839] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:18.416384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:18.424956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c2e05-e08f-478b-b819-983a83bff846 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.463716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209041-1d9b-4261-81ea-888c8b7533f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.465048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209041-1d9b-4261-81ea-888c8b7533f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.465553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209041-1d9b-4261-81ea-888c8b7533f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.466196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3209041-1d9b-4261-81ea-888c8b7533f0 None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/1ecd34d877d4476d86e0e8b4ae9c56c9/roles/cc93d9b4856a4b1cbf172a716c54860a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.481631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:18.484723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3209041-1d9b-4261-81ea-888c8b7533f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=1ecd34d877d4476d86e0e8b4ae9c56c9, role_id=cc93d9b4856a4b1cbf172a716c54860a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.486270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3209041-1d9b-4261-81ea-888c8b7533f0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.501745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 419/840] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:06:18 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/1ecd34d877d4476d86e0e8b4ae9c56c9/roles/cc93d9b4856a4b1cbf172a716c54860a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:06:18.502626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209041-1d9b-4261-81ea-888c8b7533f0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:18.545201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044af26-61bf-4279-b83d-653e3744c7d1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.583965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:18.589151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.590249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.590612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.590895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.591684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:18.591998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.594439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.601781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 422/841] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:18.608110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:18.675497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3209041-1d9b-4261-81ea-888c8b7533f0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.719148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.719829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.720110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.720388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.721111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:18.721416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.722867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.738038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 420/842] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:18.743972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:18.759387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3df534-f27b-4500-a65e-ee1d75bd4309 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.783854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:18.810548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.811616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.811904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.812337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.813394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:18.813788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.815691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.823046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 423/843] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:18.866713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751eaf64-a517-475d-a2d8-a9e635ad06e8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:18.875362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:18.908608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:18.909435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:18.909744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:18.910028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:18.919936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:18.921491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:18.937703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:18.938951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:18.939909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 421/844] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:18.945319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:19.068301 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893eb29-a28c-4681-ad53-6a2882dc12de None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:19.107428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:19.108215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:19.108498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:19.108771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:19.109621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:19.109963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:19.111446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:19.132874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 424/845] 10.222.0.71 () {56 vars in 1135 bytes} [Mon Jul 18 14:06:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:19.139027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:19.253635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c35ec-62c7-461f-acd1-c08d8f819a09 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:19.289315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc46df-6be5-4403-9eab-47c6fa3b4393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:19.290317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc46df-6be5-4403-9eab-47c6fa3b4393 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:19.290604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc46df-6be5-4403-9eab-47c6fa3b4393 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:19.290881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc46df-6be5-4403-9eab-47c6fa3b4393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:19.330402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:19.367263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc46df-6be5-4403-9eab-47c6fa3b4393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:19.369483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc46df-6be5-4403-9eab-47c6fa3b4393 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:19.475096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 422/846] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:19.481150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:19.528422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:19.590733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7defb-7cdb-4277-9606-be46ab33bbe1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:19.627594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:19.629129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:19.629567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:19.630011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:19.644959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:19.646468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:19.665457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:19.666489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:19.668034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:19.668034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:19.668034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:19.668034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:19.668034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 425/847] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:19.674133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc46df-6be5-4403-9eab-47c6fa3b4393 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:19.754072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:19.792643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc46df-6be5-4403-9eab-47c6fa3b4393 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:19.830628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:19.831308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:19.831588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:19.831863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:19.832588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:19.832892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:19.834291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:19.847878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 423/848] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:19.853621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:19.957414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:19.969143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb35ca0-f4c0-41f5-a0f2-01105e2bb474 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:20.006983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:20.007786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.008065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.008396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.015780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:20.017300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:20.032599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:20.033593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:20.034531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 426/849] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:20.041511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:20.159935 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cdc4a-aea1-4189-b452-f22cd5956cd6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:20.194620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:20.205920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a297de-7e97-4332-8d84-218ffa01aeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:20.207081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a297de-7e97-4332-8d84-218ffa01aeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.207627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a297de-7e97-4332-8d84-218ffa01aeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.208115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a297de-7e97-4332-8d84-218ffa01aeb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.209156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a297de-7e97-4332-8d84-218ffa01aeb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:20.209700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a297de-7e97-4332-8d84-218ffa01aeb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:20.211550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a297de-7e97-4332-8d84-218ffa01aeb6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:20.226533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 424/850] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:20.233235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:20.258922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ebd89-38b7-44c1-9fd1-9819cf44d497 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.259660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ebd89-38b7-44c1-9fd1-9819cf44d497 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.260230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ebd89-38b7-44c1-9fd1-9819cf44d497 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.262495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 425/851] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:20.270790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df7fa-62c4-415e-b28b-0a08f0785fbc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.271561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df7fa-62c4-415e-b28b-0a08f0785fbc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.272145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383df7fa-62c4-415e-b28b-0a08f0785fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-383df7fa-62c4-415e-b28b-0a08f0785fbc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:20.292925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-383df7fa-62c4-415e-b28b-0a08f0785fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:20.361435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b8fc7-7d11-4c0d-a6c0-76cf1424354e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:20.401053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:20.401767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.402054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.402334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.403182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=203e654f95bf45e683b0ccc4ba687bfc) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:20.403503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:20.405057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:20.412150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 427/852] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:20 2022] GET /identity/v3/role_assignments?user.id=203e654f95bf45e683b0ccc4ba687bfc&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=5c672a1b2979444383c587a215adeb35 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:20.506940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 426/853] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:20.529172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:20.549448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:20.653358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0be920-4661-48cc-95a7-225e2d0f871e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:20.690934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:20.691668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.691944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.692212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.692862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:20.694271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:20.708749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e98b48-366d-4dd5-9b0a-3562b25df944 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.709070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e98b48-366d-4dd5-9b0a-3562b25df944 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.709300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e98b48-366d-4dd5-9b0a-3562b25df944 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:20.710340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:20.711302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:20.712322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:20.713122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 427/854] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:20.715362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 428/855] 10.222.0.71 () {56 vars in 1132 bytes} [Mon Jul 18 14:06:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:20.716472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d34f-bb95-4627-9cff-7afbfe1cb48f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.716755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d34f-bb95-4627-9cff-7afbfe1cb48f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.717023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935d34f-bb95-4627-9cff-7afbfe1cb48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.724409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:20.737437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3935d34f-bb95-4627-9cff-7afbfe1cb48f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:20.739132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3935d34f-bb95-4627-9cff-7afbfe1cb48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:20.838478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:20.846025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 428/856] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:20.863594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ea6fb-0a6d-410b-ba5a-b6c4f1b51c81 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:20.865234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d34f-bb95-4627-9cff-7afbfe1cb48f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:20.902635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:20.903671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:20.904147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:20.904612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:20.905610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:20.906148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:20.907905 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:20.912413 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:20.914918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 429/857] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:20.923206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:20.976847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935d34f-bb95-4627-9cff-7afbfe1cb48f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:21.018668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b897e-405a-4ab3-9c18-5e07d8bbb86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:21.019488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b897e-405a-4ab3-9c18-5e07d8bbb86f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.019771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b897e-405a-4ab3-9c18-5e07d8bbb86f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.020047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b897e-405a-4ab3-9c18-5e07d8bbb86f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.033632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8b897e-405a-4ab3-9c18-5e07d8bbb86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-cc8b897e-405a-4ab3-9c18-5e07d8bbb86f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:21.042042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:21.043040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:21.043903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 429/858] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:06:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:21.045682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edd2ab-5526-44c0-9abe-8d5e7719fb07 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:21.048910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b897e-405a-4ab3-9c18-5e07d8bbb86f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:21.080990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b78050-1af3-40b7-9297-dda489bf7405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:21.081694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78050-1af3-40b7-9297-dda489bf7405 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.081966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78050-1af3-40b7-9297-dda489bf7405 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.082245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b78050-1af3-40b7-9297-dda489bf7405 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.083033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b78050-1af3-40b7-9297-dda489bf7405 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:21.083344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b78050-1af3-40b7-9297-dda489bf7405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:21.084733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b78050-1af3-40b7-9297-dda489bf7405 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:21.098028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 430/859] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:20 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:21.134550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a1b26-2590-4ae5-9893-7755bfa1f109 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.135018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a1b26-2590-4ae5-9893-7755bfa1f109 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.135132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a1b26-2590-4ae5-9893-7755bfa1f109 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.141156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 431/860] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:21.143206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883038-6cf5-4f9f-a037-9e088c21cdf7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.143496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883038-6cf5-4f9f-a037-9e088c21cdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.143777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90883038-6cf5-4f9f-a037-9e088c21cdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.165409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-90883038-6cf5-4f9f-a037-9e088c21cdf7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:21.167040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-90883038-6cf5-4f9f-a037-9e088c21cdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:21.170204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:21.176891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8b897e-405a-4ab3-9c18-5e07d8bbb86f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:21.187588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:21.234221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:21.234827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.235113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.235392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.236161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:21.236472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:21.237945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:21.248216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 430/861] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:06:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:21.253358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:21.275966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 432/862] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:21.299813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90883038-6cf5-4f9f-a037-9e088c21cdf7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:21.369856 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf95cb-2e70-413b-b558-10dfe7fe6b3b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:21.415865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:21.416761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.417311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.417759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.423296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90883038-6cf5-4f9f-a037-9e088c21cdf7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:21.434945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:21.436815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:21.461713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:21.462704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:21.463741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:21.463741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:21.463741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:21.463741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:21.463741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 431/863] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:21.470744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:21.472034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4cf2f-7639-4666-9cad-ccf8c20013a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:21.472876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4cf2f-7639-4666-9cad-ccf8c20013a3 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.473281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4cf2f-7639-4666-9cad-ccf8c20013a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.474012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4cf2f-7639-4666-9cad-ccf8c20013a3 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.475920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4cf2f-7639-4666-9cad-ccf8c20013a3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:21.477721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4cf2f-7639-4666-9cad-ccf8c20013a3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:21.494112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:21.500114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 433/864] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:06:21 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:06:21.513699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:21.615363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ddb0a-092b-496b-a3b0-0878d02b907d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:21.661917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:21.662808 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.663184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.663554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.664407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:21.664913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:21.666637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:21.680592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 432/865] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:21.686682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4cf2f-7639-4666-9cad-ccf8c20013a3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:21.793698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4cf2f-7639-4666-9cad-ccf8c20013a3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:21.830624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:21.831604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:21.831980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:21.832352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:21.841081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:21.842672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:21.858482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:21.859489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:21.861480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 434/866] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:06:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:21.865818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:21.972098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:22.007319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb48f6-3e21-4972-be6e-c9db3dc26708 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:22.046719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:22.047329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:22.047701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:22.047863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:22.048702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:22.049016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:22.050469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:22.069305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 433/867] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 18 14:06:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:22.075693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:22.141398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:22.187051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbe53a-597e-46e3-8623-1c2430075b55 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:22.224142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d1091-fdea-47e9-b0b2-fad529cbe2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:22.224812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d1091-fdea-47e9-b0b2-fad529cbe2fd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:22.225157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d1091-fdea-47e9-b0b2-fad529cbe2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:22.225438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d1091-fdea-47e9-b0b2-fad529cbe2fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:22.312784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d1091-fdea-47e9-b0b2-fad529cbe2fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:22.314960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d1091-fdea-47e9-b0b2-fad529cbe2fd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:22.387568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:22.433971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 435/868] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:22 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:22.439607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:22.546981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df85fc4-dadc-4f24-a60e-d9a85bd46f06 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:22.586334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b869a-7056-4c40-be41-e3d58a722503 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:22.587502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b869a-7056-4c40-be41-e3d58a722503 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:22.587788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b869a-7056-4c40-be41-e3d58a722503 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:22.588723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:22.589531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:22.589637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 436/869] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:22.589837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:22.590121 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:22.592298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:22.594875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73c5da-6e9a-4166-8c24-5f5474a24a66 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:22.595163 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73c5da-6e9a-4166-8c24-5f5474a24a66 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:22.595441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb73c5da-6e9a-4166-8c24-5f5474a24a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:22.602445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:22.603931 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:22.615175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-eb73c5da-6e9a-4166-8c24-5f5474a24a66 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:22.616876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-eb73c5da-6e9a-4166-8c24-5f5474a24a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:22.623065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:22.624014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:22.625075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:22.625075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:22.625075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:22.625075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:22.625075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 434/870] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:22.637490 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:22.717943 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 437/871] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:22.735456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73c5da-6e9a-4166-8c24-5f5474a24a66 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:22.781979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8dcf08-d06d-428c-9cc0-6e0b04fd91cf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:22.823477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:22.824732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:22.825229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:22.825766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:22.826783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:22.827288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:22.829032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:22.836548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:22.845583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 435/872] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:22.850730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:22.856270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb73c5da-6e9a-4166-8c24-5f5474a24a66 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:22.893760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:22.894581 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:22.894865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:22.895141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:22.895799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:22.897193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:22.906314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:22.910952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:22.912112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:22.913006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:22.917961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 438/873] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:22.919608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:22.977319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf63d7b-4068-414b-a4fa-f144993e9a21 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:23.026950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:23.027827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.028715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.029020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.039419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:23.041445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:23.042380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c15a3-6790-49b5-bd62-7db2f58d1caf None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:23.061519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:23.062545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:23.063431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:23.063431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:23.063431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:23.063431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:23.063431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:23.063431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:23.063431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:23.063431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:23.067672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 436/874] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:23.068631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:23.084924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:23.085601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.085913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.086199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.087091 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:23.087369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:23.088793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:23.094548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 439/875] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:23.101840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:23.128855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:23.203773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a1c9a-ea00-4789-b6d1-660033914df7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:23.213440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:23.230071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57d51-f32f-493a-9bf3-af6feaecb764 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:23.246693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b9ad5-b3d3-4df9-91cb-a6737d4429e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:23.247366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b9ad5-b3d3-4df9-91cb-a6737d4429e7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.247650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b9ad5-b3d3-4df9-91cb-a6737d4429e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.247923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b9ad5-b3d3-4df9-91cb-a6737d4429e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.248719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b9ad5-b3d3-4df9-91cb-a6737d4429e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:23.249027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b9ad5-b3d3-4df9-91cb-a6737d4429e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:23.250497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b9ad5-b3d3-4df9-91cb-a6737d4429e7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:23.263898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 437/876] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:23.268869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffaa85-3f96-43f1-8fc9-e29a76b2340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:23.269507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffaa85-3f96-43f1-8fc9-e29a76b2340d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.269823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffaa85-3f96-43f1-8fc9-e29a76b2340d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.270099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffaa85-3f96-43f1-8fc9-e29a76b2340d None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.270227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567aa35-4cd3-496c-b89a-236385baa734 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.270475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567aa35-4cd3-496c-b89a-236385baa734 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.270755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567aa35-4cd3-496c-b89a-236385baa734 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.270852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffaa85-3f96-43f1-8fc9-e29a76b2340d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:23.271104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffaa85-3f96-43f1-8fc9-e29a76b2340d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:23.276451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 438/877] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:23.276659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffaa85-3f96-43f1-8fc9-e29a76b2340d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:23.276758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567aa35-4cd3-496c-b89a-236385baa734 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:23.289740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 440/878] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:23.289740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2429b-de1f-4cac-b625-e9e83081debf None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.289740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2429b-de1f-4cac-b625-e9e83081debf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.289740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2429b-de1f-4cac-b625-e9e83081debf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.311402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-79f2429b-de1f-4cac-b625-e9e83081debf None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:23.313074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-79f2429b-de1f-4cac-b625-e9e83081debf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:23.426424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 441/879] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:23 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:23.437496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567aa35-4cd3-496c-b89a-236385baa734 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:23.449071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2429b-de1f-4cac-b625-e9e83081debf None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:23.462059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:23.505914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a3089-d5eb-44af-8d34-5f003c484ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:23.507150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3089-d5eb-44af-8d34-5f003c484ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.507639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3089-d5eb-44af-8d34-5f003c484ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.508166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3089-d5eb-44af-8d34-5f003c484ce2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.509963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a3089-d5eb-44af-8d34-5f003c484ce2 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=1ecd34d877d4476d86e0e8b4ae9c56c9) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:23.510532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a3089-d5eb-44af-8d34-5f003c484ce2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:23.513279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a3089-d5eb-44af-8d34-5f003c484ce2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:23.533781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 439/880] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:23 2022] GET /identity/v3/role_assignments?user.id=1ecd34d877d4476d86e0e8b4ae9c56c9&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:23.581025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2429b-de1f-4cac-b625-e9e83081debf None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:23.623786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f040e8b-d11e-4fd5-9626-119f84867812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:23.624819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f040e8b-d11e-4fd5-9626-119f84867812 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.625320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f040e8b-d11e-4fd5-9626-119f84867812 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.625833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f040e8b-d11e-4fd5-9626-119f84867812 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.627209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f040e8b-d11e-4fd5-9626-119f84867812 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:23.633704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f040e8b-d11e-4fd5-9626-119f84867812 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:23.660405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 442/881] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:06:23 2022] POST /identity/v3/services => generated 269 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:06:23.772799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e57e3-5352-4bee-8f77-afbef726071a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.773014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e57e3-5352-4bee-8f77-afbef726071a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.773276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e57e3-5352-4bee-8f77-afbef726071a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.775457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 440/882] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:23.781436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8b5af-594f-4820-9c87-be7ab893cf9a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:23.782235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8b5af-594f-4820-9c87-be7ab893cf9a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:23.782235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8b5af-594f-4820-9c87-be7ab893cf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:23.804575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a0d8b5af-594f-4820-9c87-be7ab893cf9a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:23.806058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a0d8b5af-594f-4820-9c87-be7ab893cf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:23.890270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:23.903785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 443/883] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:23.927638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e57e3-5352-4bee-8f77-afbef726071a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:23.963124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:24.046724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e57e3-5352-4bee-8f77-afbef726071a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:24.083664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:24.084498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:24.084786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:24.085065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:24.085753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:24.087358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:24.100489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:24.101794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:24.103399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 441/884] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:24.108498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8b5af-594f-4820-9c87-be7ab893cf9a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:24.219284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8b5af-594f-4820-9c87-be7ab893cf9a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:24.259187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:24.259862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:24.260142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:24.260417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:24.261145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:24.261449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:24.262889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:24.270728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 444/885] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:24.276705 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:24.392138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:24.400142 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14533c99-8f1d-4ca6-8ccf-4386b0ce4a5c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:24.441882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:24.442566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:24.442852 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:24.443126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:24.443842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:24.444151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:24.445538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:24.452201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 442/886] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:24.459277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:24.536144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:24.586020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976f7c3-56f4-4944-8a3b-62fc9c361f07 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:24.625952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f2de9-2d9e-4713-93c3-36fbd9634b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:24.626626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f2de9-2d9e-4713-93c3-36fbd9634b1b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:24.626908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f2de9-2d9e-4713-93c3-36fbd9634b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:24.627184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f2de9-2d9e-4713-93c3-36fbd9634b1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:24.628028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f2de9-2d9e-4713-93c3-36fbd9634b1b None admin] RBAC: Adding query filter params (interface=public, service_id=42d6921c2a264120a73db35e4f9d8a1f, region_id=RegionOne) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:24.628341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f2de9-2d9e-4713-93c3-36fbd9634b1b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:24.631952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f2de9-2d9e-4713-93c3-36fbd9634b1b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:24.638910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 445/887] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 18 14:06:24 2022] GET /identity/v3/endpoints?service_id=42d6921c2a264120a73db35e4f9d8a1f&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:24.644044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:24.760190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad8694-1f91-4e0d-a8a1-45e9e09ec20b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:24.796746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb3f5c-2c88-4341-ac24-f9f253c11f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:24.797427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3f5c-2c88-4341-ac24-f9f253c11f94 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:24.797730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3f5c-2c88-4341-ac24-f9f253c11f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:24.798008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb3f5c-2c88-4341-ac24-f9f253c11f94 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:24.798700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb3f5c-2c88-4341-ac24-f9f253c11f94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:24.800080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cb3f5c-2c88-4341-ac24-f9f253c11f94 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:24.807227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 443/888] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:06:24 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:06:25.447884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd16e0-2a17-4239-b0aa-2366d5c0dc94 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.448425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd16e0-2a17-4239-b0aa-2366d5c0dc94 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.448425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd16e0-2a17-4239-b0aa-2366d5c0dc94 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:25.450159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 446/889] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:25.455837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657249e-178a-4beb-9806-842fc890cbe8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.456130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657249e-178a-4beb-9806-842fc890cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.456412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657249e-178a-4beb-9806-842fc890cbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:25.475242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6657249e-178a-4beb-9806-842fc890cbe8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:25.476470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6657249e-178a-4beb-9806-842fc890cbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:25.497097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:25.554329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 444/890] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:25.578324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd16e0-2a17-4239-b0aa-2366d5c0dc94 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:25.695225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd16e0-2a17-4239-b0aa-2366d5c0dc94 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:25.721792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:25.735889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e6eab-430c-4ac9-b5c6-a6150ad54fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:25.736678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e6eab-430c-4ac9-b5c6-a6150ad54fbb None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.736959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e6eab-430c-4ac9-b5c6-a6150ad54fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.737232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e6eab-430c-4ac9-b5c6-a6150ad54fbb None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:25.738036 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e6eab-430c-4ac9-b5c6-a6150ad54fbb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:25.739862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e6eab-430c-4ac9-b5c6-a6150ad54fbb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:25.756481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 447/891] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:06:25 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:25.892263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57368-d75b-449a-85c5-7dc0f1fcb6c4 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.892966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57368-d75b-449a-85c5-7dc0f1fcb6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.892966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a57368-d75b-449a-85c5-7dc0f1fcb6c4 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:25.894765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 445/892] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:25.901618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33d2db-475b-4c0f-96b7-dba46f8129c2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.902164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33d2db-475b-4c0f-96b7-dba46f8129c2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.902643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33d2db-475b-4c0f-96b7-dba46f8129c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:25.925634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ee33d2db-475b-4c0f-96b7-dba46f8129c2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:25.927691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ee33d2db-475b-4c0f-96b7-dba46f8129c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:25.970985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e420f-154c-446e-af90-bc6ec4d9c071 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.971324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e420f-154c-446e-af90-bc6ec4d9c071 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.971536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e420f-154c-446e-af90-bc6ec4d9c071 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:25.973334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 446/893] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:25.979076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69132-9154-409f-9b94-046ffa851a99 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:25.979751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69132-9154-409f-9b94-046ffa851a99 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:25.980245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e69132-9154-409f-9b94-046ffa851a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.002653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-74e69132-9154-409f-9b94-046ffa851a99 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:26.003252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-74e69132-9154-409f-9b94-046ffa851a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:26.033813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 448/894] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:26.050560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33d2db-475b-4c0f-96b7-dba46f8129c2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.101385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 447/895] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:26.129229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e69132-9154-409f-9b94-046ffa851a99 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.176629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33d2db-475b-4c0f-96b7-dba46f8129c2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.189899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:26.219698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407d970-daed-4aad-aac5-970c9c6f0d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.220903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407d970-daed-4aad-aac5-970c9c6f0d4f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.221378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407d970-daed-4aad-aac5-970c9c6f0d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.221934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407d970-daed-4aad-aac5-970c9c6f0d4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.231268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:26.234085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407d970-daed-4aad-aac5-970c9c6f0d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8407d970-daed-4aad-aac5-970c9c6f0d4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:26.244065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:26.245114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:26.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:26.253693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e69132-9154-409f-9b94-046ffa851a99 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.254133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 449/896] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:26.255597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407d970-daed-4aad-aac5-970c9c6f0d4f None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.297457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.298367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.298651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.298927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.299587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.301633 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:26.317649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:26.318723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:26.319555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 448/897] 10.222.0.71 () {56 vars in 1132 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:26.326784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.376238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407d970-daed-4aad-aac5-970c9c6f0d4f None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.412178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.413055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.413433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.414202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.414853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:26.415260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.416758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.435742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 450/898] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:26.438084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.441493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2490e1-d13c-409c-b748-ebaffd9838fd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.471786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:26.480749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.481750 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.482195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.482615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.483494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:26.484050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.485642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.496794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 449/899] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:26.498200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.512638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:26.548533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe11f0f7-e70f-44a9-8144-e4f24b5b997b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.583462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.584403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.584588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.584827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.596251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.597809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.613025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e6c3a-5b6c-4e0f-b3d1-d4860dda8af9 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:26.616428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:26.617361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:26.618885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:26.618885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:26.618885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:26.618885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:26.622301 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 451/900] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:26.623586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.655095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.656179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.656667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.657166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.658219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:26.658754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.660383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.671470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 450/901] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:26.672774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.735428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:26.753722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086942ef-ee5e-4ee7-8a8b-b578f59c71c1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.759232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:26.789196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.789862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943669b-9093-4b65-9bcb-131f018436c3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.790088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.790505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.790787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.791507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:26.791810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.793189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.811528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 452/902] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:26.812917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.828784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.829868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.830334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.830787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.831811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] RBAC: Adding query filter params (interface=public, service_id=114384d615c347c6a84d1892a8d978d9, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:26.832333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.834190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.845024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 451/903] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/endpoints?service_id=114384d615c347c6a84d1892a8d978d9&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:26.846414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.918475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111aa3a-e77d-4ffd-b379-2ac90f913f54 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.954315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:26.955058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:26.955366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:26.955731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:26.958187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660321ac-07c1-405e-b051-41b939a048ba None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:26.964185 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:26.965864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:26.983140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:26.984171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:26.985111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:26.989284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 453/904] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:26.990652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:26.995021 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.003294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a022b7-7f64-4e7e-a374-250411a28062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.004000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a022b7-7f64-4e7e-a374-250411a28062 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.004255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a022b7-7f64-4e7e-a374-250411a28062 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.004532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a022b7-7f64-4e7e-a374-250411a28062 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.005225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a022b7-7f64-4e7e-a374-250411a28062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.006634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a022b7-7f64-4e7e-a374-250411a28062 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.013558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 452/905] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:06:27.023314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48639a4c-b544-4d13-84a2-148990b9b462 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.023903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48639a4c-b544-4d13-84a2-148990b9b462 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.024169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48639a4c-b544-4d13-84a2-148990b9b462 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.025967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 453/906] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:27.031720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a291a3e-c93f-4c73-81cb-565af10629f5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.032162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a291a3e-c93f-4c73-81cb-565af10629f5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.032364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a291a3e-c93f-4c73-81cb-565af10629f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.044816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.063152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4a291a3e-c93f-4c73-81cb-565af10629f5 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:27.065399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4a291a3e-c93f-4c73-81cb-565af10629f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:27.131708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7136a1-e6f8-4b0b-ba44-739e0c05c6c4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:27.153180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 454/907] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:27.176166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a291a3e-c93f-4c73-81cb-565af10629f5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:27.193152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.194056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.194346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.194714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.195764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:27.196079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.197831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.218451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 454/908] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 18 14:06:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:27.226081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:27.296253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a291a3e-c93f-4c73-81cb-565af10629f5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:27.306059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.328254 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.333650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.334836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.335609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.336566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.337137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.338934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.349496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7441a-6d7a-4dde-a5df-cf16fa65d6e7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:27.352995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:27.354240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 18 14:06:27.355127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:27.356242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 455/909] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:27.361928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:27.396503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c44e-19f1-4f19-ad4a-bc7a9632723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.397564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c44e-19f1-4f19-ad4a-bc7a9632723d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.398067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c44e-19f1-4f19-ad4a-bc7a9632723d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.398512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026c44e-19f1-4f19-ad4a-bc7a9632723d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.475809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ceeed2-bad9-49b9-a1c4-1cf9427b04b2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:27.491315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c44e-19f1-4f19-ad4a-bc7a9632723d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.494343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026c44e-19f1-4f19-ad4a-bc7a9632723d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.513268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.514319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.514479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.514901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.515683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:27.516072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.517604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.528695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 456/910] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:27.529879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:27.574942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.599217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.638007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 455/911] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:27 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 14:06:27.639052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c44e-19f1-4f19-ad4a-bc7a9632723d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:27.655952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737a692-6a57-4526-a0c8-b8dfb5ea46d3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:27.690923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.691590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.691920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.692199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.692930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:27.693239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.694896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.701029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 457/912] 10.222.0.71 () {56 vars in 1114 bytes} [Mon Jul 18 14:06:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:27.707817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:27.769371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026c44e-19f1-4f19-ad4a-bc7a9632723d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:27.805080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.805958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.806237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.806508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.817340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.818962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.831181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88a870-957c-4643-a180-b678701b56bd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:27.837958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:27.838917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:27.839791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:27.839791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:27.839791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:27.839791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:27.844317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 456/913] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:27.845259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:27.862554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.864030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:27.879479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17835e1f-5ee6-4a1c-94e8-6cdfe02a6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:27.881044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17835e1f-5ee6-4a1c-94e8-6cdfe02a6b1f None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.881511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17835e1f-5ee6-4a1c-94e8-6cdfe02a6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.881985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17835e1f-5ee6-4a1c-94e8-6cdfe02a6b1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.883622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17835e1f-5ee6-4a1c-94e8-6cdfe02a6b1f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:27.885459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17835e1f-5ee6-4a1c-94e8-6cdfe02a6b1f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:27.908545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 458/914] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:06:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:06:27.919338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33618580-4b02-430d-8c2c-26c6c090715b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.919681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33618580-4b02-430d-8c2c-26c6c090715b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.919905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33618580-4b02-430d-8c2c-26c6c090715b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.921482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 459/915] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:27.927108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7dc44-8763-4a05-aa78-d2e8e84dcc91 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:27.927405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7dc44-8763-4a05-aa78-d2e8e84dcc91 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:27.927659 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7dc44-8763-4a05-aa78-d2e8e84dcc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:27.953027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-47c7dc44-8763-4a05-aa78-d2e8e84dcc91 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:27.955132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-47c7dc44-8763-4a05-aa78-d2e8e84dcc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:27.967891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045fd48-f12b-4e26-a4ee-770ddf989eb6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.014946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.015626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.015907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.016180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.016907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:28.017214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.018731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.032275 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 457/916] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:28.038441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:28.060857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 460/917] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:28.086051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7dc44-8763-4a05-aa78-d2e8e84dcc91 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:28.120201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:28.169027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75122266-0ec4-4cde-a833-f704f7d72396 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.204607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7dc44-8763-4a05-aa78-d2e8e84dcc91 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.209479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.210323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.210608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.210885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.218728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.219061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:28.220309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:28.236873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:28.238076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:28.239204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 458/918] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:28.245998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:28.252299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.253219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.253565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.253964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.254695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.256579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:28.269735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:28.270810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:28.271648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 461/919] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:28.278205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:28.370132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b238388-f761-4fb1-9d7a-91246bb60a76 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.406226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092e86c-d21d-4d5e-9d55-4358e111c8c3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.410911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:28.418711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.420182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.420614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.421065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.422398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:28.422873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.425112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.440746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 459/920] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:28.446996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:28.459338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.460128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.460325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.460607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.461399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:28.461742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.463394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.477645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 462/921] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:28.479275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:28.540743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:28.573980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716e544-b73a-416a-8893-2e6b7537bf85 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.612562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.613392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.613894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.614220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.615167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] RBAC: Adding query filter params (role.id=ffc0e4716c434657a52d2a5b235f99e8, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=1ecd34d877d4476d86e0e8b4ae9c56c9) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:28.615585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.617139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.618739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901602d-5c9a-4fc1-b412-55a973f68b9a None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.624881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 460/922] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:28 2022] GET /identity/v3/role_assignments?user.id=1ecd34d877d4476d86e0e8b4ae9c56c9&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=ffc0e4716c434657a52d2a5b235f99e8 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:28.659657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.660348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.660630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.660906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.661631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:28.661969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.663378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.670007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 463/923] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:28.675957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:28.742100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:28.792488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657cccb0-89b3-4ac3-970b-57aae48b87a1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.828277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d16241-bc07-464e-a6b8-c56ada06d908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.828942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16241-bc07-464e-a6b8-c56ada06d908 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.829223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16241-bc07-464e-a6b8-c56ada06d908 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.829499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16241-bc07-464e-a6b8-c56ada06d908 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.830327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d16241-bc07-464e-a6b8-c56ada06d908 None admin] RBAC: Adding query filter params (interface=public, service_id=6d7e2096046e4e42bfe9c74627c0dae9, region_id=RegionOne) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:28.830636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d16241-bc07-464e-a6b8-c56ada06d908 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.832012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d16241-bc07-464e-a6b8-c56ada06d908 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.838181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 461/924] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 18 14:06:28 2022] GET /identity/v3/endpoints?service_id=6d7e2096046e4e42bfe9c74627c0dae9&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:28.843744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:28.948226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4887e9-ed8a-4abd-a8a3-27ff7f3889ce None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:28.949793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:28.983380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23ec58-7eea-4ba0-b9ab-34572ceafd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:28.984078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ec58-7eea-4ba0-b9ab-34572ceafd4a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:28.984365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ec58-7eea-4ba0-b9ab-34572ceafd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:28.984643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23ec58-7eea-4ba0-b9ab-34572ceafd4a None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:28.985341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23ec58-7eea-4ba0-b9ab-34572ceafd4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:28.987369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23ec58-7eea-4ba0-b9ab-34572ceafd4a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:28.995783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 464/925] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:06:28 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:06:29.141758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40d50-c1f8-4117-8215-2d793c4964ab None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:29.142013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40d50-c1f8-4117-8215-2d793c4964ab None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:29.142290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b40d50-c1f8-4117-8215-2d793c4964ab None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:29.144224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 462/926] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:29.149882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b677-cd36-42d1-8978-0d4b9c09cfd6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:29.150431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b677-cd36-42d1-8978-0d4b9c09cfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:29.150431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b677-cd36-42d1-8978-0d4b9c09cfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:29.176070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1022b677-cd36-42d1-8978-0d4b9c09cfd6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:29.177282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1022b677-cd36-42d1-8978-0d4b9c09cfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:29.257842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 465/927] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:29.276460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b40d50-c1f8-4117-8215-2d793c4964ab None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:29.383234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b40d50-c1f8-4117-8215-2d793c4964ab None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:29.402079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:29.420449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:29.421633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:29.422161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:29.422624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:29.423485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:29.425076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:29.439007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:29.440007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 18 14:06:29.440857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:29.449342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 463/928] 10.222.0.71 () {56 vars in 1132 bytes} [Mon Jul 18 14:06:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:29.451003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022b677-cd36-42d1-8978-0d4b9c09cfd6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:29.552523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:29.560580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022b677-cd36-42d1-8978-0d4b9c09cfd6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:29.597834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:29.598963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:29.599441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:29.599966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:29.600870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:29.601369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:29.603751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:29.613790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 466/929] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:29.616711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:29.752777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53156711-17b3-4f7e-8c5a-5740d1bea5ff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:29.787796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ea103-fd51-4c54-aa03-0afe8b5582e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:29.788477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ea103-fd51-4c54-aa03-0afe8b5582e0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:29.788764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ea103-fd51-4c54-aa03-0afe8b5582e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:29.789038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ea103-fd51-4c54-aa03-0afe8b5582e0 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:29.790144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ea103-fd51-4c54-aa03-0afe8b5582e0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:29.790780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ea103-fd51-4c54-aa03-0afe8b5582e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:29.792176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ea103-fd51-4c54-aa03-0afe8b5582e0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:29.801797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 464/930] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:29.803945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:29.911922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b6516-f66e-421c-bda1-b577251c22c3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:29.945969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a8df7-2aad-41e2-b1f3-4cb354a55552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:29.947070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a8df7-2aad-41e2-b1f3-4cb354a55552 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:29.947559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a8df7-2aad-41e2-b1f3-4cb354a55552 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:29.948038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a8df7-2aad-41e2-b1f3-4cb354a55552 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:29.949062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a8df7-2aad-41e2-b1f3-4cb354a55552 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:29.950678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a8df7-2aad-41e2-b1f3-4cb354a55552 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:29.970782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 467/931] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:06:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:30.138445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f03a2-3dff-4f84-be53-77eb2b2354eb None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:30.138773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f03a2-3dff-4f84-be53-77eb2b2354eb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:30.139018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f03a2-3dff-4f84-be53-77eb2b2354eb None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:30.140408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 465/932] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:30.145868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41d8b5-5746-418d-9fdc-0c15f3a27816 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:30.146165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41d8b5-5746-418d-9fdc-0c15f3a27816 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:30.146448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41d8b5-5746-418d-9fdc-0c15f3a27816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:30.164574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ff41d8b5-5746-418d-9fdc-0c15f3a27816 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:30.165819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ff41d8b5-5746-418d-9fdc-0c15f3a27816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:30.177588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:30.257075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 468/933] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:30.280676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f03a2-3dff-4f84-be53-77eb2b2354eb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:30.333817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:30.393189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f03a2-3dff-4f84-be53-77eb2b2354eb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:30.429949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:30.431234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:30.431520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:30.431793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:30.432510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:30.433993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:30.435387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING keystone.server.flask.common [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 14:06:30.455765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.sql.core [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 14:06:30.455765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 14:06:30.455765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [parameters: {'id': '5b8410c55afb44d39e0f33b318d7b035', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 14:06:30.455765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80214) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5b8410c55afb44d39e0f33b318d7b035.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5b8410c55afb44d39e0f33b318d7b035. Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:06:30.473709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application result.read() Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 18 14:06:30.474632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application yield resource Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise self.value Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application session.commit() Jul 18 14:06:30.475612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.session.flush() Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application compat.raise_( Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 14:06:30.476505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 14:06:30.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application result.read() Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:06:30.478684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [parameters: {'id': '5b8410c55afb44d39e0f33b318d7b035', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:30.479571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:30.480342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5b8410c55afb44d39e0f33b318d7b035. Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:30.481252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 466/934] 10.222.0.71 () {60 vars in 1140 bytes} [Mon Jul 18 14:06:30 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:30.486236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41d8b5-5746-418d-9fdc-0c15f3a27816 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:30.591273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41d8b5-5746-418d-9fdc-0c15f3a27816 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:30.629084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd27ca-f951-49c5-8645-3fb1e9c0f599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:30.630318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd27ca-f951-49c5-8645-3fb1e9c0f599 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:30.630837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd27ca-f951-49c5-8645-3fb1e9c0f599 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:30.631317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd27ca-f951-49c5-8645-3fb1e9c0f599 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:30.655116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd27ca-f951-49c5-8645-3fb1e9c0f599 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:30.656540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd27ca-f951-49c5-8645-3fb1e9c0f599 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e5fd27ca-f951-49c5-8645-3fb1e9c0f599 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:30.671992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:30.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:30.673863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 469/935] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:30.679440 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:30.767515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4cf0c-1cad-4bcd-84c7-9f06060d648f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:30.768015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4cf0c-1cad-4bcd-84c7-9f06060d648f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:30.768504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4cf0c-1cad-4bcd-84c7-9f06060d648f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:30.775623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 470/936] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:30.778233 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4632760-912b-4326-a181-6f9e1677680a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:30.778520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4632760-912b-4326-a181-6f9e1677680a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:30.778872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4632760-912b-4326-a181-6f9e1677680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:30.785026 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:30.797504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6dcfe-97ee-4e7a-9b61-f65553a6b62d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:30.799814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:30.800875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c4632760-912b-4326-a181-6f9e1677680a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:30.804383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c4632760-912b-4326-a181-6f9e1677680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:30.842114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:30.843183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:30.843695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:30.844277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:30.845350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:30.845902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:30.847930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:30.873154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 467/937] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:30.892828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 471/938] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:30.916382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.034677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957cd53-12d2-4d61-bc5c-5f3b7c092d76 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.071180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e18cc36-e807-4d68-a8c7-520243f24532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:31.072006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18cc36-e807-4d68-a8c7-520243f24532 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.072287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18cc36-e807-4d68-a8c7-520243f24532 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.072560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e18cc36-e807-4d68-a8c7-520243f24532 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.083468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e18cc36-e807-4d68-a8c7-520243f24532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8e18cc36-e807-4d68-a8c7-520243f24532 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:31.091658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:31.092650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:31.093485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 468/939] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 18 14:06:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:31.099307 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4632760-912b-4326-a181-6f9e1677680a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.209289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4632760-912b-4326-a181-6f9e1677680a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.242696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:31.249894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:31.250583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.250863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.251136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.251395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e254c-c0cc-4af0-9a0c-010e37eccbc2 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.251689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e254c-c0cc-4af0-9a0c-010e37eccbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.251892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:31.251993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e254c-c0cc-4af0-9a0c-010e37eccbc2 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.252197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:31.253569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:31.253702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 469/940] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:31.259895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a225-70f2-4ffd-9458-e4af16f43de0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.260336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a225-70f2-4ffd-9458-e4af16f43de0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.260737 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7a225-70f2-4ffd-9458-e4af16f43de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.264614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 472/941] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 18 14:06:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:31.270697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.286359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bdb7a225-70f2-4ffd-9458-e4af16f43de0 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:31.288834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bdb7a225-70f2-4ffd-9458-e4af16f43de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:31.315396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:31.373851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 470/942] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:31.382159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c384ba2-99e4-4a01-8d35-177ee75b7efd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.396059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7a225-70f2-4ffd-9458-e4af16f43de0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.422381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:31.423197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.423484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.423758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.435488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:31.436955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:31.454410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:31.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:31.456825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:31.456825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:31.456825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:31.456825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:31.456825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 473/943] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:31.462064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.520242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:31.526582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7a225-70f2-4ffd-9458-e4af16f43de0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.563314 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:31.564134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.564413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.564687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.565346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:31.566761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:31.581140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598caef-124d-4698-bb7e-a1ec35138999 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:31.587302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:31.588570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:31.589616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 471/944] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:31.598771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.622964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:31.623685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:31.625213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.625716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.626200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.627222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:31.627764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:31.630001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:31.658373 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 474/945] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:31.661007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.729814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491ab39-975a-41cf-a787-7ba5788b5cfc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.779179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:31.780239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.780729 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9814e75-eed3-464f-83a5-5d7d13ff11ce None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.780973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.781460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.782656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:31.784057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:31.785708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:31.788895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:31.798251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 472/946] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:31.799303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.823855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:31.824694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.824982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.825315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.833218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:31.834716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:31.850711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:31.851788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:31.852744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:31.856826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 475/947] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:06:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:31.857879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:31.912172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:31.938644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2e0ee-2969-4e5a-80b4-7cb631581cee None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.966149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f631b06-3e56-4df6-b3ca-a702f863cf32 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:31.975938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:31.977072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:31.977596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:31.978109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:31.979051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:31.979568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:31.981219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:31.991938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 473/948] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:31.993554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.004194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.004711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.005002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.005277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.006158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:32.006480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.008177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.027991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 476/949] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 18 14:06:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:32.033632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.060777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:32.111329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccddfa81-9852-44be-af54-ea375ea2e330 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:32.148601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb383df-e297-4082-a532-9788b9ca4486 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:32.150346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f001ac-fbc7-49e9-bfd5-cc90a6b24fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.151291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f001ac-fbc7-49e9-bfd5-cc90a6b24fc3 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.151804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f001ac-fbc7-49e9-bfd5-cc90a6b24fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.152301 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f001ac-fbc7-49e9-bfd5-cc90a6b24fc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.153267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f001ac-fbc7-49e9-bfd5-cc90a6b24fc3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.155059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f001ac-fbc7-49e9-bfd5-cc90a6b24fc3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.171383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:32.186080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 474/950] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:06:31 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:32.192186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79641c0d-80e9-4da3-9a1a-eb7f888a9115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.193025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c0d-80e9-4da3-9a1a-eb7f888a9115 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.193316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c0d-80e9-4da3-9a1a-eb7f888a9115 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.193594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641c0d-80e9-4da3-9a1a-eb7f888a9115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.207407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79641c0d-80e9-4da3-9a1a-eb7f888a9115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.209197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79641c0d-80e9-4da3-9a1a-eb7f888a9115 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.228931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501350ba-59b6-4732-9e1e-a2d56f1069a0 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.229253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501350ba-59b6-4732-9e1e-a2d56f1069a0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.229726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501350ba-59b6-4732-9e1e-a2d56f1069a0 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-79641c0d-80e9-4da3-9a1a-eb7f888a9115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:32.229888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:32.231106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:32.232083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:32.232083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:32.232083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:32.232083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:32.232083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 477/952] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:32.232414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 475/952] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:32.236920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501350ba-59b6-4732-9e1e-a2d56f1069a0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.237369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97ad39-67b3-4dfb-8002-24f2015a4fba None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.237678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97ad39-67b3-4dfb-8002-24f2015a4fba None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.237968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97ad39-67b3-4dfb-8002-24f2015a4fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.256539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7c97ad39-67b3-4dfb-8002-24f2015a4fba None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:32.257824 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7c97ad39-67b3-4dfb-8002-24f2015a4fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:32.331879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 478/953] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:32.353846 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97ad39-67b3-4dfb-8002-24f2015a4fba None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.356184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:32.368367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501350ba-59b6-4732-9e1e-a2d56f1069a0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:32.414948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.415848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.416086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.416368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.417103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:32.417544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.418857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.432841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 476/954] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:32.439182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.480616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97ad39-67b3-4dfb-8002-24f2015a4fba None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:32.517520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.518640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.519090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.519543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.520067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:32.520862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.522594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:32.536439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:32.537882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:32.538996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:32.545690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 479/955] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:32.548169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.572257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1c0878-aae4-4d42-9019-a89737af4753 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:32.617440 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.618918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.619310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.619689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.621956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:32.628557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.630925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:32.656122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:32.657081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:32.658638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:32.658638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:32.658638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:32.658638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:32.658638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:32.658638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:32.658638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:32.658638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:32.661994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 477/956] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:32.666953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.679523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595313cf-9235-42f4-a2a9-1f33ecb09c2e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:32.726222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.727388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.727879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.728835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.729873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:32.730639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.732434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.740319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 480/957] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:32.746267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.789034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da00d66-3f06-477e-b407-38de34e107f6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:32.815828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:32.834932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.835706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.836142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.836563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.837506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:32.837975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.839977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.860210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 478/958] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:32.861426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:32.871738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b341d-cf56-47bf-9622-aa4dc8376ef1 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:32.908349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:32.915774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec1444c-20d2-41b3-b07f-9d64e869ae6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:32.916889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1444c-20d2-41b3-b07f-9d64e869ae6e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:32.917359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1444c-20d2-41b3-b07f-9d64e869ae6e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:32.917976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec1444c-20d2-41b3-b07f-9d64e869ae6e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:32.918806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1444c-20d2-41b3-b07f-9d64e869ae6e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:32.919331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1444c-20d2-41b3-b07f-9d64e869ae6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:32.921121 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec1444c-20d2-41b3-b07f-9d64e869ae6e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:32.933823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 481/959] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:32.980248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68b9a2-06ff-402f-b0db-d80bc8bc624a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:33.021676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe2a158-48db-428d-8f08-221cb995c274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:33.022814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2a158-48db-428d-8f08-221cb995c274 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:33.023198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2a158-48db-428d-8f08-221cb995c274 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:33.023686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe2a158-48db-428d-8f08-221cb995c274 None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/1ecd34d877d4476d86e0e8b4ae9c56c9/roles/ffc0e4716c434657a52d2a5b235f99e8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:33.026500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2a158-48db-428d-8f08-221cb995c274 None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=1ecd34d877d4476d86e0e8b4ae9c56c9, role_id=ffc0e4716c434657a52d2a5b235f99e8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:33.028167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe2a158-48db-428d-8f08-221cb995c274 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:33.046556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 479/960] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:06:32 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/1ecd34d877d4476d86e0e8b4ae9c56c9/roles/ffc0e4716c434657a52d2a5b235f99e8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:06:33.144996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1547-b8e3-4830-9c42-b0f1a1705cd0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:33.145292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1547-b8e3-4830-9c42-b0f1a1705cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:33.145590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f1547-b8e3-4830-9c42-b0f1a1705cd0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:33.147314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 482/961] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:33.153121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97032e-953d-4b22-8832-c2f77104e9e1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:33.153412 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97032e-953d-4b22-8832-c2f77104e9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:33.153727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97032e-953d-4b22-8832-c2f77104e9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:33.173748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7c97032e-953d-4b22-8832-c2f77104e9e1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:33.175303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7c97032e-953d-4b22-8832-c2f77104e9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:33.216862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:33.260562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 480/962] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:33.284707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f1547-b8e3-4830-9c42-b0f1a1705cd0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:33.425692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f1547-b8e3-4830-9c42-b0f1a1705cd0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:33.467656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:33.468766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:33.469195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:33.469637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:33.470556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:33.472217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:33.473781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING keystone.server.flask.common [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 18 14:06:33.488891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.sql.core [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 14:06:33.488891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 14:06:33.488891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [parameters: {'user_id': '50099375adfe43ffad3dc294ef246cfa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 14:06:33.488891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80213) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result.read() Jul 18 14:06:33.499801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:06:33.500948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application yield resource Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application next(self.gen) Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.session.rollback() Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.force_reraise() Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise self.value Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application session.commit() Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 18 14:06:33.502402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._prepare_impl() Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.session.flush() Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._flush(objects) Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application compat.raise_( Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application flush_context.execute() Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rec.execute(self) Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 18 14:06:33.503456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 18 14:06:33.504411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result = self._query(query) Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application conn.query(q) Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application result.read() Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 18 14:06:33.505425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [parameters: {'user_id': '50099375adfe43ffad3dc294ef246cfa', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:33.506378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 18 14:06:33.507244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 18 14:06:33.508361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 18 14:06:33.508361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 18 14:06:33.508361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 18 14:06:33.508361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 18 14:06:33.508361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 18 14:06:33.508361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:33.512034 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 483/963] 10.222.0.71 () {60 vars in 1131 bytes} [Mon Jul 18 14:06:33 2022] POST /identity/v3/users => generated 160 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:33.518329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97032e-953d-4b22-8832-c2f77104e9e1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:33.629562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97032e-953d-4b22-8832-c2f77104e9e1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:33.669407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d87fd-8044-430e-a07f-23ff45d0b63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:33.670556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d87fd-8044-430e-a07f-23ff45d0b63c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:33.670975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d87fd-8044-430e-a07f-23ff45d0b63c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:33.671400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d87fd-8044-430e-a07f-23ff45d0b63c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:33.679383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d87fd-8044-430e-a07f-23ff45d0b63c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:33.681422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d87fd-8044-430e-a07f-23ff45d0b63c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-b69d87fd-8044-430e-a07f-23ff45d0b63c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:33.697097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:33.698415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 18 14:06:33.699706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:33.702030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 481/964] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:33.708634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:33.771523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:33.823939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34640d6-1d4e-43fc-8684-a89b350b37e7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:33.859576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee834a7-5268-44f6-bc1c-908e78d735b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:33.865298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee834a7-5268-44f6-bc1c-908e78d735b1 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:33.865298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee834a7-5268-44f6-bc1c-908e78d735b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:33.865298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee834a7-5268-44f6-bc1c-908e78d735b1 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:33.865298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee834a7-5268-44f6-bc1c-908e78d735b1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:33.865298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee834a7-5268-44f6-bc1c-908e78d735b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:33.865298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee834a7-5268-44f6-bc1c-908e78d735b1 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:33.910136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 484/965] 10.222.0.71 () {56 vars in 1101 bytes} [Mon Jul 18 14:06:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:34.316872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59112c-cd2d-468a-88bf-82e887e38230 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:34.317474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59112c-cd2d-468a-88bf-82e887e38230 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:34.317474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59112c-cd2d-468a-88bf-82e887e38230 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:34.318984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 482/966] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:34.324647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543baec-1b57-451c-b4e9-f685816f87d2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:34.325099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543baec-1b57-451c-b4e9-f685816f87d2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:34.325212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543baec-1b57-451c-b4e9-f685816f87d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:34.343316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0543baec-1b57-451c-b4e9-f685816f87d2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:34.344829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0543baec-1b57-451c-b4e9-f685816f87d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:34.424886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 485/967] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:34.447712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59112c-cd2d-468a-88bf-82e887e38230 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:34.481632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:34.564738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59112c-cd2d-468a-88bf-82e887e38230 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:34.611297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:34.612334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:34.612625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:34.612901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:34.613615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:34.615053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:34.636384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:34.637527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:34.638424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 483/968] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:34.647658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543baec-1b57-451c-b4e9-f685816f87d2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:34.704800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:34.762306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543baec-1b57-451c-b4e9-f685816f87d2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:34.798178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:34.798862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:34.799142 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:34.799417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:34.800130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:34.800438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:34.801839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:34.807013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 486/969] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:34.812893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:34.916659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:34.925055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a8421-cc25-4b27-9a2c-c8a726bf9c5c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:34.959555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39d503-246d-46de-aa6c-5f800d4bbc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:34.960368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39d503-246d-46de-aa6c-5f800d4bbc5f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:34.960765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39d503-246d-46de-aa6c-5f800d4bbc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:34.961175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39d503-246d-46de-aa6c-5f800d4bbc5f None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:34.962102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39d503-246d-46de-aa6c-5f800d4bbc5f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:34.962504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39d503-246d-46de-aa6c-5f800d4bbc5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:34.963983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39d503-246d-46de-aa6c-5f800d4bbc5f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:34.969839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 484/970] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:34.975652 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:35.056718 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc84d58-c078-4b8e-81d1-7dacdfb4932b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.056987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc84d58-c078-4b8e-81d1-7dacdfb4932b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.057280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc84d58-c078-4b8e-81d1-7dacdfb4932b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.058696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 485/971] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:35.063925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d0d6e-87df-4b66-ac8b-92fbf183a878 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.064215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d0d6e-87df-4b66-ac8b-92fbf183a878 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.064496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d0d6e-87df-4b66-ac8b-92fbf183a878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.081231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-234d0d6e-87df-4b66-ac8b-92fbf183a878 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:35.082453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-234d0d6e-87df-4b66-ac8b-92fbf183a878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:35.086485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9ff34-2ee3-479f-aa73-76e285ef4d0d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:35.096596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:35.124459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:35.125265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.125836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.126009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.126848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] RBAC: Adding query filter params (interface=admin, service_id=6d7e2096046e4e42bfe9c74627c0dae9, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:35.127227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:35.128709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:35.134462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 487/972] 10.222.0.71 () {56 vars in 1249 bytes} [Mon Jul 18 14:06:34 2022] GET /identity/v3/endpoints?service_id=6d7e2096046e4e42bfe9c74627c0dae9&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:35.140574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:35.160172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 486/973] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:35.161516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f15b89-d787-44de-a824-0d2b6ad6ea0c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.161896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f15b89-d787-44de-a824-0d2b6ad6ea0c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.162112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f15b89-d787-44de-a824-0d2b6ad6ea0c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.163396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 487/974] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:35.168620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5b653-2c90-4272-9fc1-e03cb7d2c7e9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.168849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5b653-2c90-4272-9fc1-e03cb7d2c7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.169130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5b653-2c90-4272-9fc1-e03cb7d2c7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.186340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e5c5b653-2c90-4272-9fc1-e03cb7d2c7e9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:35.187550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e5c5b653-2c90-4272-9fc1-e03cb7d2c7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:35.251773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5dd606-0ea4-4cca-a4a2-b95d07b35a11 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:35.257518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 488/975] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:35.258529 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5b653-2c90-4272-9fc1-e03cb7d2c7e9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:35.277808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:35.292929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0103a-ef01-45ea-ba2f-2c7712f6a689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:35.293780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0103a-ef01-45ea-ba2f-2c7712f6a689 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.294063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0103a-ef01-45ea-ba2f-2c7712f6a689 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.294437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0103a-ef01-45ea-ba2f-2c7712f6a689 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.295021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0103a-ef01-45ea-ba2f-2c7712f6a689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:35.296456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0103a-ef01-45ea-ba2f-2c7712f6a689 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:35.302828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 488/976] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:06:35 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:06:35.303679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0103a-ef01-45ea-ba2f-2c7712f6a689 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:35.343312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:35.375853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5b653-2c90-4272-9fc1-e03cb7d2c7e9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:35.418750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fa8f1-f03d-440f-952d-c9c4b61a7ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:35.419326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fa8f1-f03d-440f-952d-c9c4b61a7ca7 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.419620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fa8f1-f03d-440f-952d-c9c4b61a7ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.419918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fa8f1-f03d-440f-952d-c9c4b61a7ca7 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.420634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fa8f1-f03d-440f-952d-c9c4b61a7ca7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:35.422113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fa8f1-f03d-440f-952d-c9c4b61a7ca7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:35.422327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0103a-ef01-45ea-ba2f-2c7712f6a689 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:35.438538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 489/977] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:06:35 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:35.477564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39360cf-9063-4435-8ac0-20c292a8cbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:35.478587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39360cf-9063-4435-8ac0-20c292a8cbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.478958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39360cf-9063-4435-8ac0-20c292a8cbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.479331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39360cf-9063-4435-8ac0-20c292a8cbb2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.491364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39360cf-9063-4435-8ac0-20c292a8cbb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-c39360cf-9063-4435-8ac0-20c292a8cbb2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:35.500525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:35.501772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:35.502772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 489/978] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 18 14:06:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:35.508248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fa8f1-f03d-440f-952d-c9c4b61a7ca7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:35.615228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:35.647633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fa8f1-f03d-440f-952d-c9c4b61a7ca7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:35.686568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:35.687166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.687445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.687724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.688487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:35.688799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:35.690248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:35.700290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 490/979] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 18 14:06:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:35.706457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39360cf-9063-4435-8ac0-20c292a8cbb2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:35.812362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:35.826887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39360cf-9063-4435-8ac0-20c292a8cbb2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:35.863357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:35.864160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:35.864442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:35.864713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:35.877230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:35.878832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:35.896979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:35.898208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:35.899880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:35.899880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:35.899880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:35.899880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:35.899880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 490/980] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:35.904448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:36.011346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9319a9f-d3b8-4e07-bafa-ddd779f17b0c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:36.050445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:36.051564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:36.052042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:36.052544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:36.053923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:36.054419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:36.056598 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:36.066496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:36.075766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 491/981] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:36.077277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:36.184759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5eb47-812a-4257-9815-70ac1ecfc378 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:36.220839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:36.221822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:36.222169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:36.222508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:36.230617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:36.232152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:36.247367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:36.248566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 18 14:06:36.249817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:36.250959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 491/982] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:06:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:36.256519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:36.365759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e3541-68eb-4806-a6f2-24e92f9baccc None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:36.402068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:36.402736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:36.402998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:36.403278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:36.404106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:36.404408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:36.405870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:36.435911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 492/983] 10.222.0.71 () {56 vars in 1149 bytes} [Mon Jul 18 14:06:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:36.441466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:36.446593 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:36.557601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd0d5e-5681-4397-88c9-b5a38ab0329d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:36.594583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704585e8-db60-4569-b463-0e116e1650c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:36.595400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704585e8-db60-4569-b463-0e116e1650c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:36.595809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704585e8-db60-4569-b463-0e116e1650c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:36.596183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704585e8-db60-4569-b463-0e116e1650c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:36.677475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704585e8-db60-4569-b463-0e116e1650c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:36.678969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704585e8-db60-4569-b463-0e116e1650c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:36.682651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:36.800578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 492/984] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:36 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:36.806705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:36.873464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:36.920016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444e087-b4d7-4978-af36-4442be80485d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:36.957745 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:36.958544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:36.958822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:36.959093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:36.969963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:36.971438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:36.989305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:36.990514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:36.991508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:36.991508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:36.991508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:36.991508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:36.992364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 493/985] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:36.997079 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704585e8-db60-4569-b463-0e116e1650c3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.113692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704585e8-db60-4569-b463-0e116e1650c3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:37.116148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:37.157016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:37.159491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.159491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.159491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.159491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:37.159491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:37.161104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:37.175447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 493/986] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:37.181032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.305600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d494b8-4989-49be-8f5b-73d864b81ed9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:37.317737 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:37.344295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee093bb0-a05e-42a0-827c-48493e227c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:37.345106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee093bb0-a05e-42a0-827c-48493e227c5b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.345383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee093bb0-a05e-42a0-827c-48493e227c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.345665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee093bb0-a05e-42a0-827c-48493e227c5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.353252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee093bb0-a05e-42a0-827c-48493e227c5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:37.354701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee093bb0-a05e-42a0-827c-48493e227c5b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ee093bb0-a05e-42a0-827c-48493e227c5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:37.369754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:37.370725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:37.371710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 494/987] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:37.378618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.457706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214884ca-9381-46dd-9bf9-f2f52b216789 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.458239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214884ca-9381-46dd-9bf9-f2f52b216789 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.458239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214884ca-9381-46dd-9bf9-f2f52b216789 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.459766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 495/988] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:37.465709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd4961-e70f-4473-9337-2aab92601751 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.466041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd4961-e70f-4473-9337-2aab92601751 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.466269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd4961-e70f-4473-9337-2aab92601751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.483805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-36cd4961-e70f-4473-9337-2aab92601751 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:37.485024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-36cd4961-e70f-4473-9337-2aab92601751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:37.499348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0ecd0-6536-4d34-b838-350e02b669a5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:37.537105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac00e3c-4bfa-40f7-aa93-dbcfe07e3dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:37.537906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac00e3c-4bfa-40f7-aa93-dbcfe07e3dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.538102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac00e3c-4bfa-40f7-aa93-dbcfe07e3dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.538379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac00e3c-4bfa-40f7-aa93-dbcfe07e3dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.539352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac00e3c-4bfa-40f7-aa93-dbcfe07e3dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:37.539628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac00e3c-4bfa-40f7-aa93-dbcfe07e3dd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:37.541189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac00e3c-4bfa-40f7-aa93-dbcfe07e3dd2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:37.556204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 494/989] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:37.557582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163a5b5-25fb-441f-ba1f-1554b564d314 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.558004 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163a5b5-25fb-441f-ba1f-1554b564d314 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.558283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163a5b5-25fb-441f-ba1f-1554b564d314 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.559581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 495/990] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:37.562321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163a5b5-25fb-441f-ba1f-1554b564d314 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.565446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 496/991] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:37.566920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343938fe-8252-4052-96fc-7dd2b7b784a7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.567214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343938fe-8252-4052-96fc-7dd2b7b784a7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.567492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343938fe-8252-4052-96fc-7dd2b7b784a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.585064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-343938fe-8252-4052-96fc-7dd2b7b784a7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:37.586383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-343938fe-8252-4052-96fc-7dd2b7b784a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:37.591143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:37.659996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 497/992] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:37.661088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343938fe-8252-4052-96fc-7dd2b7b784a7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.681917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163a5b5-25fb-441f-ba1f-1554b564d314 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:37.698703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:37.718365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:37.718937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.719220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.719499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.720338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] RBAC: Adding query filter params (role.id=ffc0e4716c434657a52d2a5b235f99e8, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=1ecd34d877d4476d86e0e8b4ae9c56c9) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:37.720661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:37.722139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:37.728721 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 496/993] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:37 2022] GET /identity/v3/role_assignments?user.id=1ecd34d877d4476d86e0e8b4ae9c56c9&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=ffc0e4716c434657a52d2a5b235f99e8 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:37.729547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.784816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343938fe-8252-4052-96fc-7dd2b7b784a7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:37.827547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:37.828379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.828760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.829040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.829762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:37.831313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:37.839573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:37.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:37.847861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:37.848751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 498/994] 10.222.0.71 () {56 vars in 1126 bytes} [Mon Jul 18 14:06:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:37.855097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.864500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44e69-691f-4128-ad13-27b642fa9893 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:37.919338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:37.920189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:37.920460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:37.920739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:37.921393 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:37.922880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:37.936610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:37.937807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:37.938649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 497/995] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:37.947202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:37.990293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:38.002390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4e310-e7bc-466c-812c-8883060df589 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:38.045022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:38.045908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.046223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.046511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.047232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:38.047540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:38.048955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:38.056032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 499/996] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:38.061881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:38.068525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b825ec-8571-44cc-b618-bc790fcb8a52 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:38.114476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:38.115161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.115454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.115724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.116446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:38.116740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:38.118738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:38.127872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 498/997] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:38.136877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:38.161782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:38.202811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a520e5-752c-40c2-b8b7-d3eec5ebbffa None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:38.241869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:38.242702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.243470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.244014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.244478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:38.244782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:38.246208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:38.256763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 500/998] 10.222.0.71 () {56 vars in 1122 bytes} [Mon Jul 18 14:06:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:38.257944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:38.259110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db307140-cbbd-43ad-81e0-91432687ecf4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:38.286174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:38.294986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:38.295739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.295953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.296224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.296939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] RBAC: Adding query filter params (type=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:38.297245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:38.299193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:38.304925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 499/999] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:38.310905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:38.370224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c91ec-8370-4878-bd76-52dbbfc28f61 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:38.396069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:38.405928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12b23f-0843-48f8-b0bf-59c4b447712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:38.406588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12b23f-0843-48f8-b0bf-59c4b447712c None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.406870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12b23f-0843-48f8-b0bf-59c4b447712c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.407145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12b23f-0843-48f8-b0bf-59c4b447712c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.407875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12b23f-0843-48f8-b0bf-59c4b447712c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:38.409258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12b23f-0843-48f8-b0bf-59c4b447712c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:38.419108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f468d-e08c-4e7d-8267-c2489290f8ad None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:38.429392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 501/1000] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:06:38 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:38.465199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c46cbf-0f08-4ad4-806b-64f23acde797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:38.465989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c46cbf-0f08-4ad4-806b-64f23acde797 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.466270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c46cbf-0f08-4ad4-806b-64f23acde797 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.466583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c46cbf-0f08-4ad4-806b-64f23acde797 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.467454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c46cbf-0f08-4ad4-806b-64f23acde797 None admin] RBAC: Adding query filter params (interface=public, service_id=035186ec01f1469d9f49021989a55cbf, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:38.467823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c46cbf-0f08-4ad4-806b-64f23acde797 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:38.469229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c46cbf-0f08-4ad4-806b-64f23acde797 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:38.480206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 500/1001] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 18 14:06:38 2022] GET /identity/v3/endpoints?service_id=035186ec01f1469d9f49021989a55cbf&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:38.482049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12b23f-0843-48f8-b0bf-59c4b447712c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:38.584778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:38.606491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12b23f-0843-48f8-b0bf-59c4b447712c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:38.647936 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fd5eb-5e8d-4aed-91d8-3f1dad6215c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:38.648644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd5eb-5e8d-4aed-91d8-3f1dad6215c6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:38.648929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd5eb-5e8d-4aed-91d8-3f1dad6215c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:38.649206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd5eb-5e8d-4aed-91d8-3f1dad6215c6 None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:38.649985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fd5eb-5e8d-4aed-91d8-3f1dad6215c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:38.651374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fd5eb-5e8d-4aed-91d8-3f1dad6215c6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:38.658304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 502/1002] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:06:38 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:06:39.931609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aee794-c256-46f7-a92d-0e454c59cea9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:39.932129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aee794-c256-46f7-a92d-0e454c59cea9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:39.932129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aee794-c256-46f7-a92d-0e454c59cea9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:39.933763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 501/1003] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:39.938907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea38ff-d711-450f-acc5-68016f612ed2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:39.939336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea38ff-d711-450f-acc5-68016f612ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:39.939431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea38ff-d711-450f-acc5-68016f612ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:39.956625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-08ea38ff-d711-450f-acc5-68016f612ed2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:39.957852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-08ea38ff-d711-450f-acc5-68016f612ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:40.031160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 503/1004] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:40.053699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aee794-c256-46f7-a92d-0e454c59cea9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:40.167720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aee794-c256-46f7-a92d-0e454c59cea9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:40.203101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:40.203826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:40.204107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:40.204382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] PATH_INFO: `/v3/services/image` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:40.205039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:40.206449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:40.219088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:40.220150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:40.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:40.225529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 502/1005] 10.222.0.71 () {56 vars in 1105 bytes} [Mon Jul 18 14:06:40 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:40.226628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea38ff-d711-450f-acc5-68016f612ed2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:40.308164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:40.335454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea38ff-d711-450f-acc5-68016f612ed2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:40.372097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e0e0-cb0a-4404-9d5d-b1e43c324abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:40.372767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e0e0-cb0a-4404-9d5d-b1e43c324abb None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:40.373062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e0e0-cb0a-4404-9d5d-b1e43c324abb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:40.373336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e0e0-cb0a-4404-9d5d-b1e43c324abb None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:40.374175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602e0e0-cb0a-4404-9d5d-b1e43c324abb None admin] RBAC: Adding query filter params (name=image) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:40.374487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602e0e0-cb0a-4404-9d5d-b1e43c324abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:40.375901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602e0e0-cb0a-4404-9d5d-b1e43c324abb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:40.381808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 504/1006] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:40.388801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:40.394308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:40.467341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298346a1-df54-48ee-9b0e-6d8e5c0758b3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:40.467586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298346a1-df54-48ee-9b0e-6d8e5c0758b3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:40.467756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298346a1-df54-48ee-9b0e-6d8e5c0758b3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:40.469472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 505/1007] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:40.474817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5851b-2a30-4167-ba7a-cb58ce7c4635 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:40.475092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5851b-2a30-4167-ba7a-cb58ce7c4635 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:40.475397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5851b-2a30-4167-ba7a-cb58ce7c4635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:40.495064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-24d5851b-2a30-4167-ba7a-cb58ce7c4635 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:40.497085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-24d5851b-2a30-4167-ba7a-cb58ce7c4635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:40.506438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901da7ec-3b27-4955-9de0-fd55ff5e2494 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:40.544464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:40.545116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:40.545398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:40.545700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:40.546426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:40.546736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:40.548135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:40.553342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 503/1008] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:40.576634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 506/1009] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:40.600712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:40.708228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:40.714461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d658a16-808f-4e7f-b59c-5158ac713a79 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:40.714851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d658a16-808f-4e7f-b59c-5158ac713a79 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:40.715022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d658a16-808f-4e7f-b59c-5158ac713a79 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:40.716622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 507/1010] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:40.722496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7d0b-3de2-4e7d-929f-b8d6cba4c6a4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:40.722496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7d0b-3de2-4e7d-929f-b8d6cba4c6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:40.722815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7d0b-3de2-4e7d-929f-b8d6cba4c6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:40.743536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9f0a7d0b-3de2-4e7d-929f-b8d6cba4c6a4 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:40.744944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9f0a7d0b-3de2-4e7d-929f-b8d6cba4c6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:40.758896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc899c-ebca-4578-b481-61af970dcda0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:40.812412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa787c1-fc5d-4382-a871-7367eaae172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:40.813266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa787c1-fc5d-4382-a871-7367eaae172f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:40.813519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa787c1-fc5d-4382-a871-7367eaae172f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:40.813839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa787c1-fc5d-4382-a871-7367eaae172f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:40.815108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa787c1-fc5d-4382-a871-7367eaae172f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:40.816613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa787c1-fc5d-4382-a871-7367eaae172f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:40.818661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 504/1011] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:40.825736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa787c1-fc5d-4382-a871-7367eaae172f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:40.829009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:40.835915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 508/1012] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:40.859751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a7d0b-3de2-4e7d-929f-b8d6cba4c6a4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:40.952599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa787c1-fc5d-4382-a871-7367eaae172f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:40.981852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a7d0b-3de2-4e7d-929f-b8d6cba4c6a4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:41.004545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ea6aa-c315-46e6-b4fb-d8225534326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:41.005623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ea6aa-c315-46e6-b4fb-d8225534326f None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:41.006133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ea6aa-c315-46e6-b4fb-d8225534326f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:41.006705 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ea6aa-c315-46e6-b4fb-d8225534326f None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:41.008060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ea6aa-c315-46e6-b4fb-d8225534326f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:41.009806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ea6aa-c315-46e6-b4fb-d8225534326f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:41.015137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:41.026304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:41.027119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:41.027404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:41.027682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:41.028387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:41.029817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:41.042620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:41.043755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:41.044651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 509/1013] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:41.054218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 505/1014] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:06:40 2022] POST /identity/v3/projects => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:41.055915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:41.142506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:41.190611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc11ac-a024-4714-b9d0-ef05a373737d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:41.234948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:41.235665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:41.235952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:41.236215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:41.236958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:41.237271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:41.238914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:41.245014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 510/1015] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:41.251052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ea6aa-c315-46e6-b4fb-d8225534326f None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:41.361064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ea6aa-c315-46e6-b4fb-d8225534326f None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:41.399167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ee3f8-2173-4acd-afeb-7b73c3c23dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:41.399755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ee3f8-2173-4acd-afeb-7b73c3c23dcf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:41.400044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ee3f8-2173-4acd-afeb-7b73c3c23dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:41.400324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ee3f8-2173-4acd-afeb-7b73c3c23dcf None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:41.401044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ee3f8-2173-4acd-afeb-7b73c3c23dcf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:41.401356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ee3f8-2173-4acd-afeb-7b73c3c23dcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:41.402812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ee3f8-2173-4acd-afeb-7b73c3c23dcf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:41.408441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 506/1016] 10.222.0.71 () {56 vars in 1112 bytes} [Mon Jul 18 14:06:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:41.414746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:41.536616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceb3f3-6810-44de-a861-8f2bf1cf266c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:41.564773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:41.575452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f2aef-b713-4b5a-acf5-108c92963d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:41.576993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f2aef-b713-4b5a-acf5-108c92963d7a None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:41.577900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f2aef-b713-4b5a-acf5-108c92963d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:41.578654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f2aef-b713-4b5a-acf5-108c92963d7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:41.580230 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f2aef-b713-4b5a-acf5-108c92963d7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:41.581910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f2aef-b713-4b5a-acf5-108c92963d7a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:41.629380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 511/1017] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:06:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:42.643587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac7bc-a0c7-4285-98d6-9c4e08ba4d42 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:42.644192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac7bc-a0c7-4285-98d6-9c4e08ba4d42 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:42.644192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fac7bc-a0c7-4285-98d6-9c4e08ba4d42 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:42.646984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 507/1018] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:42.652982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1afcc-bf91-4061-8c0a-ae886c363021 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:42.653463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1afcc-bf91-4061-8c0a-ae886c363021 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:42.653742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1afcc-bf91-4061-8c0a-ae886c363021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:42.672465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f1d1afcc-bf91-4061-8c0a-ae886c363021 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:42.673583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f1d1afcc-bf91-4061-8c0a-ae886c363021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:42.757116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 512/1019] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:42 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:42.779891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fac7bc-a0c7-4285-98d6-9c4e08ba4d42 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:42.837302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:42.891201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fac7bc-a0c7-4285-98d6-9c4e08ba4d42 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:42.925216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40888f5f-3ee5-42f0-9e65-44ac99ae4d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:42.925924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888f5f-3ee5-42f0-9e65-44ac99ae4d12 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:42.926211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888f5f-3ee5-42f0-9e65-44ac99ae4d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:42.926484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888f5f-3ee5-42f0-9e65-44ac99ae4d12 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:42.927227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40888f5f-3ee5-42f0-9e65-44ac99ae4d12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:42.928608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40888f5f-3ee5-42f0-9e65-44ac99ae4d12 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:42.943452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 508/1020] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:06:42 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:43.188714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc0584-7790-4693-9b0e-350066ed6a5a None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.188895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc0584-7790-4693-9b0e-350066ed6a5a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.189169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc0584-7790-4693-9b0e-350066ed6a5a None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.190741 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 513/1021] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:43.196070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391729f-1746-4d06-b4aa-59e71fde6307 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.196346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391729f-1746-4d06-b4aa-59e71fde6307 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.196649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391729f-1746-4d06-b4aa-59e71fde6307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.214528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f391729f-1746-4d06-b4aa-59e71fde6307 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:43.215722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f391729f-1746-4d06-b4aa-59e71fde6307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:43.296699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 509/1022] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:43.321562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc0584-7790-4693-9b0e-350066ed6a5a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:43.430796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc0584-7790-4693-9b0e-350066ed6a5a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:43.443251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:43.470854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eaac82-0c9a-4bb4-b87a-d84f1d86d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:43.471658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaac82-0c9a-4bb4-b87a-d84f1d86d28f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.471939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaac82-0c9a-4bb4-b87a-d84f1d86d28f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.472212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eaac82-0c9a-4bb4-b87a-d84f1d86d28f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.473394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eaac82-0c9a-4bb4-b87a-d84f1d86d28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:43.474913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eaac82-0c9a-4bb4-b87a-d84f1d86d28f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:43.476848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 514/1023] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:43.482834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391729f-1746-4d06-b4aa-59e71fde6307 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:43.524163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:43.586503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391729f-1746-4d06-b4aa-59e71fde6307 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:43.621983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08da81-8044-4b5f-8868-3afc06dcfa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:43.622715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08da81-8044-4b5f-8868-3afc06dcfa13 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.623513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08da81-8044-4b5f-8868-3afc06dcfa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.623513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08da81-8044-4b5f-8868-3afc06dcfa13 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.623944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08da81-8044-4b5f-8868-3afc06dcfa13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:43.625391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb08da81-8044-4b5f-8868-3afc06dcfa13 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:43.661712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 510/1024] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:06:43 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:06:43.871822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5e2b-f64b-4830-a189-2975c5025ca2 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.872665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5e2b-f64b-4830-a189-2975c5025ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.872665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d5e2b-f64b-4830-a189-2975c5025ca2 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.873768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 515/1025] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:43.881175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e142fa2-c82a-4a72-a081-42ff48eed75d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:43.881421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e142fa2-c82a-4a72-a081-42ff48eed75d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:43.881605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e142fa2-c82a-4a72-a081-42ff48eed75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:43.900058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8e142fa2-c82a-4a72-a081-42ff48eed75d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:43.901298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8e142fa2-c82a-4a72-a081-42ff48eed75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:43.981197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 511/1026] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:44.004379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d5e2b-f64b-4830-a189-2975c5025ca2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:44.129013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d5e2b-f64b-4830-a189-2975c5025ca2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:44.165084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566282-73c3-4990-8218-695857799aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:44.166068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566282-73c3-4990-8218-695857799aea None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:44.166347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566282-73c3-4990-8218-695857799aea None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:44.166669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22566282-73c3-4990-8218-695857799aea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:44.177815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22566282-73c3-4990-8218-695857799aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-22566282-73c3-4990-8218-695857799aea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:44.186209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:44.187169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:44.188025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:44.190045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 516/1027] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 18 14:06:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:44.193522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e142fa2-c82a-4a72-a081-42ff48eed75d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:44.225041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:44.316245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e142fa2-c82a-4a72-a081-42ff48eed75d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:44.354122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:44.354806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:44.355093 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:44.355361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:44.356109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:44.356417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:44.357879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:44.367453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 512/1028] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 18 14:06:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:44.374289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566282-73c3-4990-8218-695857799aea None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:44.424028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:44.500758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22566282-73c3-4990-8218-695857799aea None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:44.537850 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:44.538661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:44.539181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:44.539326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:44.551208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:44.552718 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:44.571918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:44.573107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:44.574102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:44.574102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:44.574102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:44.574102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:44.578130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 517/1029] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:44.579020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:44.702054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:44.730322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc087d9-4897-44aa-b2cb-3e91fce7433d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:44.773052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78370f34-dd2d-4b1a-8031-ac8564b6e533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:44.774011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370f34-dd2d-4b1a-8031-ac8564b6e533 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:44.774515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370f34-dd2d-4b1a-8031-ac8564b6e533 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:44.774877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78370f34-dd2d-4b1a-8031-ac8564b6e533 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:44.775626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78370f34-dd2d-4b1a-8031-ac8564b6e533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:44.775946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78370f34-dd2d-4b1a-8031-ac8564b6e533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:44.777352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78370f34-dd2d-4b1a-8031-ac8564b6e533 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:44.792174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 513/1030] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:44.797970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:44.926506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:44.939993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d16c7-2b5f-45e4-9f2b-8f20f1a39728 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:44.981717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:44.982457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:44.982740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:44.983015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:44.992226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:44.993979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.004508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbab1d4-7369-4641-8d63-ff78b8f7cb5b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.004773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbab1d4-7369-4641-8d63-ff78b8f7cb5b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.005108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbab1d4-7369-4641-8d63-ff78b8f7cb5b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.009706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 514/1031] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:45.010477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:45.011491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:45.013124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 518/1032] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:06:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:45.015487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487498-3b8f-4bad-82ff-1cee48791558 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.015777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487498-3b8f-4bad-82ff-1cee48791558 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.016052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01487498-3b8f-4bad-82ff-1cee48791558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.017513 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.036466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-01487498-3b8f-4bad-82ff-1cee48791558 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:45.038408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-01487498-3b8f-4bad-82ff-1cee48791558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:45.134608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 515/1033] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:45.151575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6ec54-9879-46e1-b8e5-841a0883b983 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.174629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01487498-3b8f-4bad-82ff-1cee48791558 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.186791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:45.187478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.187764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.188042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.188888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:45.189202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:45.190717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.212777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 519/1034] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:45.218930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.232569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:45.297924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:45.344511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01487498-3b8f-4bad-82ff-1cee48791558 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.364159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae124a-c385-44a6-b7e6-58ccb0ee057c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.397147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:45.398185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.398781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.399065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] PATH_INFO: `/v3/services/image` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.399735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:45.401142 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.410565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b9131-ae45-413c-8cf5-fd00d4c73724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:45.411224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b9131-ae45-413c-8cf5-fd00d4c73724 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.411511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b9131-ae45-413c-8cf5-fd00d4c73724 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.411780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b9131-ae45-413c-8cf5-fd00d4c73724 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:45.413356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:45.415312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:45.416230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 516/1035] 10.222.0.71 () {56 vars in 1105 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/services/image => generated 86 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:45.421939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.496986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b9131-ae45-413c-8cf5-fd00d4c73724 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:45.498541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b9131-ae45-413c-8cf5-fd00d4c73724 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.511584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:45.555383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d89e99-55de-4a8a-bd97-7416821c2188 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.595957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:45.597166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.598009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.598009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.598678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:45.599068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:45.600719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.607509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 517/1036] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:45.613988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.618921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:45.629490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 520/1037] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:45.633711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b9131-ae45-413c-8cf5-fd00d4c73724 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.769505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3d3fc9-4026-4d6c-9b49-34ff0165d3e6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.779166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b9131-ae45-413c-8cf5-fd00d4c73724 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.790592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:45.806272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:45.806952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.807239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.807513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.808362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] RBAC: Adding query filter params (type=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:45.808672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:45.810082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.820148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 518/1038] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:45.821398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.828271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aabbde-e46b-4491-b81c-57d790bd558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:45.829197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabbde-e46b-4491-b81c-57d790bd558a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.829845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabbde-e46b-4491-b81c-57d790bd558a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.830124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aabbde-e46b-4491-b81c-57d790bd558a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.841617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aabbde-e46b-4491-b81c-57d790bd558a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:45.843133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aabbde-e46b-4491-b81c-57d790bd558a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-11aabbde-e46b-4491-b81c-57d790bd558a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:45.861327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:45.862382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:45.863284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:45.863284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:45.863284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:45.863284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:45.863284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 521/1039] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:45.864194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11985-e7ad-4669-986b-cbfeb39161e1 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.864565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11985-e7ad-4669-986b-cbfeb39161e1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.864770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11985-e7ad-4669-986b-cbfeb39161e1 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.866111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 522/1040] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:45.869126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c11985-e7ad-4669-986b-cbfeb39161e1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:45.897905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:45.936874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc40b1-b1cd-4a7b-a362-739398c1a017 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.975004 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f836f2-7d12-4683-a7fb-2983231b62fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:45.975677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f836f2-7d12-4683-a7fb-2983231b62fe None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.975960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f836f2-7d12-4683-a7fb-2983231b62fe None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.976368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f836f2-7d12-4683-a7fb-2983231b62fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:45.977883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f836f2-7d12-4683-a7fb-2983231b62fe None admin] RBAC: Adding query filter params (interface=public, service_id=28afbd1ae9f34bb69deaf8f1b51a558f, region_id=RegionOne) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:45.978205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f836f2-7d12-4683-a7fb-2983231b62fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:45.979622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f836f2-7d12-4683-a7fb-2983231b62fe None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:45.983878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c11985-e7ad-4669-986b-cbfeb39161e1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:45.985247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 519/1041] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/endpoints?service_id=28afbd1ae9f34bb69deaf8f1b51a558f&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:45.986591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822201c4-f6db-49eb-b3ad-6b2f033c22d2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:45.986902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822201c4-f6db-49eb-b3ad-6b2f033c22d2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:45.987149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822201c4-f6db-49eb-b3ad-6b2f033c22d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.004961 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-822201c4-f6db-49eb-b3ad-6b2f033c22d2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:46.006210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-822201c4-f6db-49eb-b3ad-6b2f033c22d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:46.029123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:46.029714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:46.029998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:46.030275 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.031003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:46.031307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:46.032705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:46.046613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 523/1042] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:46.047469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:46.070705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:46.086392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 520/1043] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:46.087247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822201c4-f6db-49eb-b3ad-6b2f033c22d2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:46.162234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:46.171019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058955be-d6a9-432f-a955-4a8c2f3b3bec None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:46.206991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822201c4-f6db-49eb-b3ad-6b2f033c22d2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:46.225869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbeb00-b72b-4d07-b266-45a5fccdd77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:46.226653 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbeb00-b72b-4d07-b266-45a5fccdd77b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:46.226940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbeb00-b72b-4d07-b266-45a5fccdd77b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:46.227224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbeb00-b72b-4d07-b266-45a5fccdd77b None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.227931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbeb00-b72b-4d07-b266-45a5fccdd77b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:46.229371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbeb00-b72b-4d07-b266-45a5fccdd77b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:46.237203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 524/1044] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:06:46 2022] GET /identity/v3/services => generated 1957 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:06:46.237823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbeb00-b72b-4d07-b266-45a5fccdd77b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:46.250465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:46.251488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:46.251796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:46.252246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.261321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:46.263844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:46.279513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:46.281299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:46.283222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 521/1045] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:46.287038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:46.366171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:46.390468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbeb00-b72b-4d07-b266-45a5fccdd77b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:46.438433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e8620-5223-4533-8c88-62e8f841a8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:46.439275 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8620-5223-4533-8c88-62e8f841a8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:46.439554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8620-5223-4533-8c88-62e8f841a8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:46.439822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e8620-5223-4533-8c88-62e8f841a8a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.440113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eebad6-d49d-41e5-b3a2-3ebd647787c3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:46.456495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e8620-5223-4533-8c88-62e8f841a8a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-f77e8620-5223-4533-8c88-62e8f841a8a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:46.464368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:46.465408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:46.466319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 525/1046] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:46.473104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e8620-5223-4533-8c88-62e8f841a8a6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:46.475802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:46.488926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:46.489610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:46.489921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:46.490311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.491003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:46.491309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:46.493938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:46.507333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 522/1047] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:46.513803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:46.594255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e8620-5223-4533-8c88-62e8f841a8a6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:46.632130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:46.633315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:46.633602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:46.633906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.634751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:46.635103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:46.637190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:46.647979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 526/1048] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:46.654763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:46.663266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb531296-a145-4bae-991f-f46911c82c09 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:46.665542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:46.700737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:46.701740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:46.702188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:46.702609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.703612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] RBAC: Adding query filter params (role.id=ffc0e4716c434657a52d2a5b235f99e8, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=4fffb91636744fc39886bdf568317ab5) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:46.703928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:46.705380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:46.717761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 523/1049] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:46 2022] GET /identity/v3/role_assignments?user.id=4fffb91636744fc39886bdf568317ab5&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=ffc0e4716c434657a52d2a5b235f99e8 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:46.835757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3adfe0-e968-48b0-b84e-91a28b1c7b00 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:46.897193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff10aae-185a-4adb-b8f7-74207ed2eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:46.898622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10aae-185a-4adb-b8f7-74207ed2eea4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:46.898962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10aae-185a-4adb-b8f7-74207ed2eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:46.899168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff10aae-185a-4adb-b8f7-74207ed2eea4 None admin] PATH_INFO: `/v3/users/033febc3766d484f98e597477f569618` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:46.914914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff10aae-185a-4adb-b8f7-74207ed2eea4 None admin] RBAC: Authorizing `identity:get_user(user_id=033febc3766d484f98e597477f569618)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:46.916585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff10aae-185a-4adb-b8f7-74207ed2eea4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:46.918546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 527/1050] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:06:46 2022] GET /identity/v3/users/033febc3766d484f98e597477f569618 => generated 285 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:46.927875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:47.014985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:47.122604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bce1dd-b07d-42d8-8f44-cbed06f3a796 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:47.183438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c6bbf-c769-4d07-aba7-472088833ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:47.184558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c6bbf-c769-4d07-aba7-472088833ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:47.184965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c6bbf-c769-4d07-aba7-472088833ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:47.185460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c6bbf-c769-4d07-aba7-472088833ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:47.274291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c6bbf-c769-4d07-aba7-472088833ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:47.275269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c6bbf-c769-4d07-aba7-472088833ff6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:47.329287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:47.409162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 524/1051] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:46 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:47.415568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff10aae-185a-4adb-b8f7-74207ed2eea4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:47.490555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:47.534584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff10aae-185a-4adb-b8f7-74207ed2eea4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:47.573041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282c2e1-e56a-4630-8c61-bbde6e9673cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:47.574090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282c2e1-e56a-4630-8c61-bbde6e9673cf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:47.574373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282c2e1-e56a-4630-8c61-bbde6e9673cf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:47.574629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282c2e1-e56a-4630-8c61-bbde6e9673cf None admin] PATH_INFO: `/v3/projects/6b0eeb2626d7481fba333867ef047f88` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:47.576025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282c2e1-e56a-4630-8c61-bbde6e9673cf None admin] RBAC: Authorizing `identity:get_project(project_id=6b0eeb2626d7481fba333867ef047f88)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:47.577545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282c2e1-e56a-4630-8c61-bbde6e9673cf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:47.579654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 528/1052] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:06:47 2022] GET /identity/v3/projects/6b0eeb2626d7481fba333867ef047f88 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:47.585964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c6bbf-c769-4d07-aba7-472088833ff6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:47.698157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c6bbf-c769-4d07-aba7-472088833ff6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:47.737128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25680abb-5f88-4a59-8236-75e1736ff188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:47.737835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25680abb-5f88-4a59-8236-75e1736ff188 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:47.738127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25680abb-5f88-4a59-8236-75e1736ff188 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:47.738405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25680abb-5f88-4a59-8236-75e1736ff188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:47.739242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25680abb-5f88-4a59-8236-75e1736ff188 None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=6b0eeb2626d7481fba333867ef047f88, user.id=033febc3766d484f98e597477f569618) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:47.739563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25680abb-5f88-4a59-8236-75e1736ff188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:47.741247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25680abb-5f88-4a59-8236-75e1736ff188 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:47.753686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 525/1053] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:47 2022] GET /identity/v3/role_assignments?user.id=033febc3766d484f98e597477f569618&scope.project.id=6b0eeb2626d7481fba333867ef047f88&role.id=5c672a1b2979444383c587a215adeb35 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:48.497546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e4904-3753-44f8-85f4-9943f0b05f02 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:48.498136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e4904-3753-44f8-85f4-9943f0b05f02 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:48.498460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e4904-3753-44f8-85f4-9943f0b05f02 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:48.500907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 529/1054] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:48.506760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63402e-7b39-40e5-b968-ea5d8b68b31f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:48.507271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63402e-7b39-40e5-b968-ea5d8b68b31f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:48.507393 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63402e-7b39-40e5-b968-ea5d8b68b31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:48.525356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3a63402e-7b39-40e5-b968-ea5d8b68b31f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:48.526602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3a63402e-7b39-40e5-b968-ea5d8b68b31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:48.530807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:48.608113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 526/1055] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:48.633150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e4904-3753-44f8-85f4-9943f0b05f02 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:48.770264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e4904-3753-44f8-85f4-9943f0b05f02 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:48.806614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:48.815753 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79619b1-4a97-4853-85db-1de3fb3544df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:48.816951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79619b1-4a97-4853-85db-1de3fb3544df None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:48.817428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79619b1-4a97-4853-85db-1de3fb3544df None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:48.817931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79619b1-4a97-4853-85db-1de3fb3544df None admin] PATH_INFO: `/v3/services/image` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:48.818862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79619b1-4a97-4853-85db-1de3fb3544df None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:48.820673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79619b1-4a97-4853-85db-1de3fb3544df None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e79619b1-4a97-4853-85db-1de3fb3544df None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:48.834690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:48.836141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:48.837089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 530/1056] 10.222.0.71 () {56 vars in 1105 bytes} [Mon Jul 18 14:06:48 2022] GET /identity/v3/services/image => generated 86 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:48.843905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63402e-7b39-40e5-b968-ea5d8b68b31f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:48.859631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e4b3-278e-445d-a97d-95a915ea9dbb None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:48.860493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e4b3-278e-445d-a97d-95a915ea9dbb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:48.860493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6e4b3-278e-445d-a97d-95a915ea9dbb None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:48.862332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 531/1057] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:48.869699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397405-a4eb-4480-9d17-7d4e22d16b3b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:48.870090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397405-a4eb-4480-9d17-7d4e22d16b3b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:48.870373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45397405-a4eb-4480-9d17-7d4e22d16b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:48.889440 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-45397405-a4eb-4480-9d17-7d4e22d16b3b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:48.890689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-45397405-a4eb-4480-9d17-7d4e22d16b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:48.970092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63402e-7b39-40e5-b968-ea5d8b68b31f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:48.998772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 532/1058] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:49.009644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:49.010488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.010802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.011068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.011790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] RBAC: Adding query filter params (name=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:49.012091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:49.013494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:49.025149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 527/1059] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:49.025351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45397405-a4eb-4480-9d17-7d4e22d16b3b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.026283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.030849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:49.144824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45397405-a4eb-4480-9d17-7d4e22d16b3b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:49.170535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd199324-6545-4702-a456-5c2503bbcbbd None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:49.175317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:49.193226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d472d5f-879f-4f75-aca6-ca77e548012e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:49.194559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d472d5f-879f-4f75-aca6-ca77e548012e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.195204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d472d5f-879f-4f75-aca6-ca77e548012e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.196497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d472d5f-879f-4f75-aca6-ca77e548012e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.215179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d472d5f-879f-4f75-aca6-ca77e548012e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:49.218179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:49.219234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.219684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.220136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.221341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] RBAC: Adding query filter params (type=image) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:49.221885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:49.224275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-5d472d5f-879f-4f75-aca6-ca77e548012e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:49.229630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:49.231267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:49.233839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 533/1060] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 18 14:06:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:49.237323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 528/1061] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:49.242808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d472d5f-879f-4f75-aca6-ca77e548012e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.245704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.364812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:49.380237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d472d5f-879f-4f75-aca6-ca77e548012e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:49.416805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede0082-accc-41fe-9ff3-f1fd44db007e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:49.444698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:49.445802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.446785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.447218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.448691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:49.449407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:49.452049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:49.465482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 534/1062] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 18 14:06:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:49.469224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17bcf8-be12-409b-9fc5-13e52d47585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:49.469947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17bcf8-be12-409b-9fc5-13e52d47585c None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.470231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17bcf8-be12-409b-9fc5-13e52d47585c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.470503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17bcf8-be12-409b-9fc5-13e52d47585c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.471273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17bcf8-be12-409b-9fc5-13e52d47585c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:49.472681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a17bcf8-be12-409b-9fc5-13e52d47585c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:49.475135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.498185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 529/1063] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:06:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:06:49.572348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:49.669025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ad15a-4e9b-4acb-8e51-b76fdc5dcac7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:49.724048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008afb25-48c7-4ed4-a258-e83e91dda294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:49.724896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008afb25-48c7-4ed4-a258-e83e91dda294 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.725164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008afb25-48c7-4ed4-a258-e83e91dda294 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.725451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008afb25-48c7-4ed4-a258-e83e91dda294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.740194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008afb25-48c7-4ed4-a258-e83e91dda294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:49.741957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008afb25-48c7-4ed4-a258-e83e91dda294 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-008afb25-48c7-4ed4-a258-e83e91dda294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:49.762408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:49.764340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:49.766271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:49.766271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:49.766271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:49.766271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:49.766271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 535/1064] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:49.776110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17bcf8-be12-409b-9fc5-13e52d47585c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.810606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c8019-72a4-44bb-950a-2ad33868b706 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.810859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c8019-72a4-44bb-950a-2ad33868b706 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.811164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c8019-72a4-44bb-950a-2ad33868b706 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.812999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 536/1065] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:06:49.818579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa859-204b-47f7-866d-83759da4ee45 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.818939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa859-204b-47f7-866d-83759da4ee45 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.819146 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fa859-204b-47f7-866d-83759da4ee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.837368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6d3fa859-204b-47f7-866d-83759da4ee45 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:49.838857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6d3fa859-204b-47f7-866d-83759da4ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:49.900894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17bcf8-be12-409b-9fc5-13e52d47585c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:49.923597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 537/1066] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:49.930747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:49.943649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:49.944343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:49.944633 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:49.945161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:49.946120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:49.946249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:49.947123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fa859-204b-47f7-866d-83759da4ee45 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.947643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:49.964197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 530/1067] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:49.970690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:49.981981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:50.073541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fa859-204b-47f7-866d-83759da4ee45 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.083452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3722630-6030-4a93-985f-fbd63107c1ff None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.111749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc7647-32ec-4163-afd1-9496be380890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.112900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc7647-32ec-4163-afd1-9496be380890 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.113462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc7647-32ec-4163-afd1-9496be380890 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.113954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc7647-32ec-4163-afd1-9496be380890 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.122766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.123587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.123871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.124147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.128521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc7647-32ec-4163-afd1-9496be380890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:50.132149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.134025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1dcc7647-32ec-4163-afd1-9496be380890 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:50.138477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:50.139505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:50.140371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:50.141835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 538/1068] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:50.146518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc7647-32ec-4163-afd1-9496be380890 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:50.150516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:50.151759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:50.152736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 531/1069] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:06:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:50.157550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:50.210709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:50.254008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:50.272698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc7647-32ec-4163-afd1-9496be380890 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.280609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dfbe64-62a7-4018-a1bd-d00b59a59205 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.322004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.322815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.323168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.323453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.323776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.324218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:50.324493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.324594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.324769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.325046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.325917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:50.326038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.326227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.327649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.339366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 539/1070] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:50.345456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:50.358973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 532/1071] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:06:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:50.360455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:50.466242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15511de1-988e-4b4b-b4a9-6116931e47d9 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.482205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6555d7c-2463-4ceb-91ba-7a693a54e840 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.483965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:50.516558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9674f1-6bb1-4de9-9118-4c772f998d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.517444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9674f1-6bb1-4de9-9118-4c772f998d3c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.517859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9674f1-6bb1-4de9-9118-4c772f998d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.518501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9674f1-6bb1-4de9-9118-4c772f998d3c None admin] PATH_INFO: `/v3/users/033febc3766d484f98e597477f569618` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.519502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9674f1-6bb1-4de9-9118-4c772f998d3c None admin] RBAC: Authorizing `identity:get_user(user_id=033febc3766d484f98e597477f569618)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.520967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9674f1-6bb1-4de9-9118-4c772f998d3c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.527629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 540/1072] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:06:50 2022] GET /identity/v3/users/033febc3766d484f98e597477f569618 => generated 285 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:50.528928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9674f1-6bb1-4de9-9118-4c772f998d3c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:50.530506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.531327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.531667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.531934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.544525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.546384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.552340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:50.566052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:50.567036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:50.567946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:50.567946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:50.567946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:50.567946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:50.567946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 533/1073] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:50.573126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:50.667188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9674f1-6bb1-4de9-9118-4c772f998d3c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.713516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371b6db-996b-4aad-9d70-ef44f40ac85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.715168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371b6db-996b-4aad-9d70-ef44f40ac85c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.715740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371b6db-996b-4aad-9d70-ef44f40ac85c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.716112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371b6db-996b-4aad-9d70-ef44f40ac85c None admin] PATH_INFO: `/v3/projects/6b0eeb2626d7481fba333867ef047f88` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.717968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49881ee9-b0a7-41f3-8b5e-49a27676d87b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.718815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371b6db-996b-4aad-9d70-ef44f40ac85c None admin] RBAC: Authorizing `identity:get_project(project_id=6b0eeb2626d7481fba333867ef047f88)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.721007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371b6db-996b-4aad-9d70-ef44f40ac85c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.724309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 541/1074] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:06:50 2022] GET /identity/v3/projects/6b0eeb2626d7481fba333867ef047f88 => generated 311 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:50.730692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371b6db-996b-4aad-9d70-ef44f40ac85c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:50.763885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.764688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.764974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.765256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.766210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:50.766523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.767935 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.787399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 534/1075] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:50.795856 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:50.813039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:50.841027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:50.860951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371b6db-996b-4aad-9d70-ef44f40ac85c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.902075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127bd80-cee5-4df8-befc-71337501dba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.904148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127bd80-cee5-4df8-befc-71337501dba5 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.904427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127bd80-cee5-4df8-befc-71337501dba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.904811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127bd80-cee5-4df8-befc-71337501dba5 None admin] PATH_INFO: `/v3/projects/6b0eeb2626d7481fba333867ef047f88/users/033febc3766d484f98e597477f569618/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.906754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127bd80-cee5-4df8-befc-71337501dba5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b0eeb2626d7481fba333867ef047f88, user_id=033febc3766d484f98e597477f569618, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.908202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0127bd80-cee5-4df8-befc-71337501dba5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.912925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef57d4b-466f-440a-ab4f-165cd0e2f0ad None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:50.924211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 542/1076] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:06:50 2022] PUT /identity/v3/projects/6b0eeb2626d7481fba333867ef047f88/users/033febc3766d484f98e597477f569618/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:06:50.959283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:50.960409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:50.960909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:50.961411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:50.970611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:50.972389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:50.989943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:50.991039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:50.992236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:50.992236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:50.992236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:50.992236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:50.992236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:50.992236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:50.992236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:50.992236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:50.995170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 535/1077] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:51.001539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127bd80-cee5-4df8-befc-71337501dba5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:51.120020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127bd80-cee5-4df8-befc-71337501dba5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:51.155698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88dfbc-9456-4532-973a-4e5d7c5ea9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:51.156359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88dfbc-9456-4532-973a-4e5d7c5ea9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:51.156637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88dfbc-9456-4532-973a-4e5d7c5ea9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:51.156904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88dfbc-9456-4532-973a-4e5d7c5ea9c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:51.157765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88dfbc-9456-4532-973a-4e5d7c5ea9c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:51.158012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88dfbc-9456-4532-973a-4e5d7c5ea9c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:51.159441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88dfbc-9456-4532-973a-4e5d7c5ea9c0 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:51.172038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 543/1078] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:51.177948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:51.262759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:51.292461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47054e6f-a5fe-45f2-bc41-ad129868bded None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:51.328547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bcf47-797f-4b1c-a79f-a2b548a5208c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:51.329461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bcf47-797f-4b1c-a79f-a2b548a5208c None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:51.329755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bcf47-797f-4b1c-a79f-a2b548a5208c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:51.330029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bcf47-797f-4b1c-a79f-a2b548a5208c None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/4fffb91636744fc39886bdf568317ab5/roles/ffc0e4716c434657a52d2a5b235f99e8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:51.332266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bcf47-797f-4b1c-a79f-a2b548a5208c None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=4fffb91636744fc39886bdf568317ab5, role_id=ffc0e4716c434657a52d2a5b235f99e8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:51.333728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bcf47-797f-4b1c-a79f-a2b548a5208c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:51.350548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 536/1079] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:06:51 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/4fffb91636744fc39886bdf568317ab5/roles/ffc0e4716c434657a52d2a5b235f99e8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:06:51.612931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77249c9-398d-4611-ae0d-6e08971d1aa3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:51.613136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77249c9-398d-4611-ae0d-6e08971d1aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:51.613371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77249c9-398d-4611-ae0d-6e08971d1aa3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:51.614987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 544/1080] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:51.620573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9949b-6992-4fe2-9289-ca4dce2fc440 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:51.620832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9949b-6992-4fe2-9289-ca4dce2fc440 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:51.621137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9949b-6992-4fe2-9289-ca4dce2fc440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:51.639330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e8e9949b-6992-4fe2-9289-ca4dce2fc440 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:51.640154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e8e9949b-6992-4fe2-9289-ca4dce2fc440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:51.719006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 537/1081] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:51.741601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77249c9-398d-4611-ae0d-6e08971d1aa3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:51.746318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:51.859215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77249c9-398d-4611-ae0d-6e08971d1aa3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:51.904952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb2998-7b1b-4939-8987-3470ab7d91bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:51.905945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb2998-7b1b-4939-8987-3470ab7d91bd None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:51.906485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb2998-7b1b-4939-8987-3470ab7d91bd None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:51.906897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb2998-7b1b-4939-8987-3470ab7d91bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:51.988455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb2998-7b1b-4939-8987-3470ab7d91bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:51.989941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb2998-7b1b-4939-8987-3470ab7d91bd None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:51.993800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:52.104508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 545/1082] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:52.110422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9949b-6992-4fe2-9289-ca4dce2fc440 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:52.181359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:52.222534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9949b-6992-4fe2-9289-ca4dce2fc440 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:52.266569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45748de9-096a-4420-af96-38d328e2550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:52.267380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748de9-096a-4420-af96-38d328e2550f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:52.267660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748de9-096a-4420-af96-38d328e2550f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:52.267930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748de9-096a-4420-af96-38d328e2550f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:52.280143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45748de9-096a-4420-af96-38d328e2550f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:52.282476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45748de9-096a-4420-af96-38d328e2550f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-45748de9-096a-4420-af96-38d328e2550f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:52.308382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:52.309607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:52.310578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:52.310578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:52.310578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:52.310578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:52.310578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 538/1083] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:52.318297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb2998-7b1b-4939-8987-3470ab7d91bd None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:52.423864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb2998-7b1b-4939-8987-3470ab7d91bd None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:52.426292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:52.462654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d336c6-696f-4403-97fa-f7c310f0d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:52.463244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d336c6-696f-4403-97fa-f7c310f0d89b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:52.463520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d336c6-696f-4403-97fa-f7c310f0d89b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:52.463792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d336c6-696f-4403-97fa-f7c310f0d89b None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:52.464515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d336c6-696f-4403-97fa-f7c310f0d89b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:52.464806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d336c6-696f-4403-97fa-f7c310f0d89b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:52.466339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d336c6-696f-4403-97fa-f7c310f0d89b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:52.479406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 546/1084] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:52.980444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a958-9ed5-48f5-a18a-c86bc41f096e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:52.980735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a958-9ed5-48f5-a18a-c86bc41f096e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:52.981010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8a958-9ed5-48f5-a18a-c86bc41f096e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:52.982469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 539/1085] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:52.987950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9da13-9f78-41ec-86ec-1efca3e79020 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:52.988456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9da13-9f78-41ec-86ec-1efca3e79020 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:52.988456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9da13-9f78-41ec-86ec-1efca3e79020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.007509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fae9da13-9f78-41ec-86ec-1efca3e79020 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:53.009405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fae9da13-9f78-41ec-86ec-1efca3e79020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:53.085922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 547/1086] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:53.110026 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8a958-9ed5-48f5-a18a-c86bc41f096e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:53.228192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8a958-9ed5-48f5-a18a-c86bc41f096e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:53.247256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:53.268567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db5656-57a8-427b-9d84-2831951ff844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:53.269381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db5656-57a8-427b-9d84-2831951ff844 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:53.269753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db5656-57a8-427b-9d84-2831951ff844 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:53.269976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db5656-57a8-427b-9d84-2831951ff844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.282557 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db5656-57a8-427b-9d84-2831951ff844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-25db5656-57a8-427b-9d84-2831951ff844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:53.290606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:53.291640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:53.292483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 540/1087] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:06:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:53.298998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9da13-9f78-41ec-86ec-1efca3e79020 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:53.401999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:53.404752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2273-f0b3-4dcb-b80b-5fd1931d8ccf None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:53.404974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2273-f0b3-4dcb-b80b-5fd1931d8ccf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:53.405277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d2273-f0b3-4dcb-b80b-5fd1931d8ccf None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.406825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 541/1088] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:53.409586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9da13-9f78-41ec-86ec-1efca3e79020 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:53.413029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058415-62bc-4c51-a287-d4698a69ba81 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:53.413315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058415-62bc-4c51-a287-d4698a69ba81 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:53.413620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058415-62bc-4c51-a287-d4698a69ba81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.431154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-03058415-62bc-4c51-a287-d4698a69ba81 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:53.432414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-03058415-62bc-4c51-a287-d4698a69ba81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:53.453542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:53.454364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:53.454650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:53.454924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.455675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:53.455985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:53.457406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:53.466506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 548/1089] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:06:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:53.472191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:53.502365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 542/1090] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 18 14:06:53.525793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03058415-62bc-4c51-a287-d4698a69ba81 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:53.578381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7224b-46b4-4219-b800-fcc1ef2e6fc7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:53.614781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07413ab-834a-481a-80ff-9aa0ee091060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:53.615646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07413ab-834a-481a-80ff-9aa0ee091060 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:53.615919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07413ab-834a-481a-80ff-9aa0ee091060 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:53.616196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07413ab-834a-481a-80ff-9aa0ee091060 None admin] PATH_INFO: `/v3/users/033febc3766d484f98e597477f569618` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.617534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07413ab-834a-481a-80ff-9aa0ee091060 None admin] RBAC: Authorizing `identity:get_user(user_id=033febc3766d484f98e597477f569618)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:53.619347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07413ab-834a-481a-80ff-9aa0ee091060 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:53.621029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 549/1091] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:06:53 2022] GET /identity/v3/users/033febc3766d484f98e597477f569618 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:53.626809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07413ab-834a-481a-80ff-9aa0ee091060 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:53.629440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03058415-62bc-4c51-a287-d4698a69ba81 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:53.638540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:53.656767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:53.669705 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b692f-1901-45f4-b4c0-cb5f41736c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:53.670537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b692f-1901-45f4-b4c0-cb5f41736c68 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:53.670828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b692f-1901-45f4-b4c0-cb5f41736c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:53.671102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b692f-1901-45f4-b4c0-cb5f41736c68 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.681477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b692f-1901-45f4-b4c0-cb5f41736c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-074b692f-1901-45f4-b4c0-cb5f41736c68 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:06:53.689622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:53.690987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:53.691840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 543/1092] 10.222.0.71 () {56 vars in 1120 bytes} [Mon Jul 18 14:06:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:53.698992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b692f-1901-45f4-b4c0-cb5f41736c68 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:53.741750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07413ab-834a-481a-80ff-9aa0ee091060 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:53.776461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5de01-e500-4810-b011-5c2e3c43f435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:53.777081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5de01-e500-4810-b011-5c2e3c43f435 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:53.777368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5de01-e500-4810-b011-5c2e3c43f435 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:53.777690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5de01-e500-4810-b011-5c2e3c43f435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.804766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b692f-1901-45f4-b4c0-cb5f41736c68 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:53.840969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:53.841697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:53.841986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:53.842261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:53.843019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:53.843328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:53.844712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:53.846120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5de01-e500-4810-b011-5c2e3c43f435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:53.847563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5de01-e500-4810-b011-5c2e3c43f435 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:53.853801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 544/1093] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 18 14:06:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:53.859542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:53.872560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:53.889120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:53.949937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 550/1094] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 14:06:53.956002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5de01-e500-4810-b011-5c2e3c43f435 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:53.961952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba18d-eff0-4581-a414-358f4e2f24bc None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:54.002665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:54.003460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:54.003710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:54.003981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:54.019165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:54.020651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:54.043802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:54.045033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:54.046197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:54.046197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:54.046197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:54.046197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:54.046197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 545/1095] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:54.050905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:54.072687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5de01-e500-4810-b011-5c2e3c43f435 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:54.095329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:54.106801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755a324-0cb3-48ed-a715-4cd82fe2a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:54.107673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a324-0cb3-48ed-a715-4cd82fe2a1ec None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:54.108090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a324-0cb3-48ed-a715-4cd82fe2a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:54.108397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a324-0cb3-48ed-a715-4cd82fe2a1ec None admin] PATH_INFO: `/v3/projects/6b0eeb2626d7481fba333867ef047f88` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:54.109642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755a324-0cb3-48ed-a715-4cd82fe2a1ec None admin] RBAC: Authorizing `identity:get_project(project_id=6b0eeb2626d7481fba333867ef047f88)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:54.111128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755a324-0cb3-48ed-a715-4cd82fe2a1ec None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:54.112853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 551/1096] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:06:53 2022] GET /identity/v3/projects/6b0eeb2626d7481fba333867ef047f88 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:54.120841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755a324-0cb3-48ed-a715-4cd82fe2a1ec None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:54.149446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:54.169424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df634c1-d1d4-4a25-81b6-6297a9bd3735 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:54.205674 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:54.206456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:54.206741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:54.207015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:54.207754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:54.208059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:54.209445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:54.222838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 546/1097] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:06:54.228859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:54.231299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755a324-0cb3-48ed-a715-4cd82fe2a1ec None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:54.269688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:54.270349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:54.270668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:54.270942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:54.271747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=6b0eeb2626d7481fba333867ef047f88, user.id=033febc3766d484f98e597477f569618) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:54.272065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:54.273835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:54.280357 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 552/1098] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:54 2022] GET /identity/v3/role_assignments?user.id=033febc3766d484f98e597477f569618&scope.project.id=6b0eeb2626d7481fba333867ef047f88&role.id=5c672a1b2979444383c587a215adeb35 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:54.350999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bb76c-2470-4c75-a760-446cd4736332 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:54.391374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:54.392201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:54.392482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:54.392761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:54.402108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:54.403600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:54.406743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:06:54.420465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:54.421567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:06:54.423035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 547/1099] 10.222.0.71 () {56 vars in 1093 bytes} [Mon Jul 18 14:06:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:06:54.427959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:54.489699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:54.538149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa371990-1819-4521-aea2-23aa1999b0da None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:54.573325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:54.574023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:54.574301 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:54.574570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:54.575399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:54.575706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:54.577135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:54.596237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 553/1100] 10.222.0.71 () {56 vars in 1133 bytes} [Mon Jul 18 14:06:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:54.602848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:54.719714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56520657-af04-40d5-83e1-1b03399e69a4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:54.755453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1fb8c-1b7e-4a4a-949d-35fe15ce7b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:54.756111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1fb8c-1b7e-4a4a-949d-35fe15ce7b7d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:54.756399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1fb8c-1b7e-4a4a-949d-35fe15ce7b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:54.756685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1fb8c-1b7e-4a4a-949d-35fe15ce7b7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:54.824360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:54.828039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1fb8c-1b7e-4a4a-949d-35fe15ce7b7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:54.829571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1fb8c-1b7e-4a4a-949d-35fe15ce7b7d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:54.930486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 548/1101] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:54.936634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:55.045048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76769a6-3463-4831-ae2d-09e918dc3b84 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:55.060136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:55.082154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:55.082953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:55.083234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:55.083506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:55.094582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:55.096046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:06:55.114431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:55.115530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:55.116606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:06:55.116606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:06:55.116606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:06:55.116606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:55.118369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 554/1102] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:55.122505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1fb8c-1b7e-4a4a-949d-35fe15ce7b7d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:55.229855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:55.233717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1fb8c-1b7e-4a4a-949d-35fe15ce7b7d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:55.270599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:55.271271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:55.271555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:55.271832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:55.276352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:55.276352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:55.276352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:55.288573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 549/1103] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:55.295900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:55.456995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b1641-46f3-43e8-a2b0-5c4ab4786bdf None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:55.496015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:55.496728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:55.496990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:55.497282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:55.505517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:55.507438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:06:55.523221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:06:55.524308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:06:55.525406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 555/1104] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:06:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:06:55.530974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:55.654030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347a68d-ebe6-4fc6-9441-803eb4935d7b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:55.689389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3ddf3-e7e2-4e54-8914-eea8b11bd2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:55.690111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3ddf3-e7e2-4e54-8914-eea8b11bd2ad None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:55.690394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3ddf3-e7e2-4e54-8914-eea8b11bd2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:55.690665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3ddf3-e7e2-4e54-8914-eea8b11bd2ad None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:55.691453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3ddf3-e7e2-4e54-8914-eea8b11bd2ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:55.691762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3ddf3-e7e2-4e54-8914-eea8b11bd2ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:55.693200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3ddf3-e7e2-4e54-8914-eea8b11bd2ad None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:55.705517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 550/1105] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:06:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:55.711977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:55.741268 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:55.819895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865653d-d9f7-4ed8-8c65-63e2d91a32ff None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:55.856311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68201eb-06e2-4445-bcb9-ad634b5dbfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:55.856971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68201eb-06e2-4445-bcb9-ad634b5dbfaa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:55.857252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68201eb-06e2-4445-bcb9-ad634b5dbfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:55.857522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68201eb-06e2-4445-bcb9-ad634b5dbfaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:55.858392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68201eb-06e2-4445-bcb9-ad634b5dbfaa None admin] RBAC: Adding query filter params (role.id=ffc0e4716c434657a52d2a5b235f99e8, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=4fffb91636744fc39886bdf568317ab5) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:06:55.858703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68201eb-06e2-4445-bcb9-ad634b5dbfaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:55.860151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68201eb-06e2-4445-bcb9-ad634b5dbfaa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:55.868421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 556/1106] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:06:55 2022] GET /identity/v3/role_assignments?user.id=4fffb91636744fc39886bdf568317ab5&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=ffc0e4716c434657a52d2a5b235f99e8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:56.368858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146c5bf-f13b-4a9d-b0e3-a3def893496a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:56.369434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146c5bf-f13b-4a9d-b0e3-a3def893496a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:56.369434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5146c5bf-f13b-4a9d-b0e3-a3def893496a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:56.370952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 551/1107] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:56.376743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c987e6-bf59-4e2d-9180-dc89932a2ddd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:56.377221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c987e6-bf59-4e2d-9180-dc89932a2ddd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:56.377313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c987e6-bf59-4e2d-9180-dc89932a2ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:56.396460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-50c987e6-bf59-4e2d-9180-dc89932a2ddd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:56.397678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-50c987e6-bf59-4e2d-9180-dc89932a2ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:56.475358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 557/1108] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:56.500788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146c5bf-f13b-4a9d-b0e3-a3def893496a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:56.524009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:56.619518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5146c5bf-f13b-4a9d-b0e3-a3def893496a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:56.662009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbf57e-c0aa-4d57-8890-3e49810ca94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:56.668746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbf57e-c0aa-4d57-8890-3e49810ca94f None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:56.668746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbf57e-c0aa-4d57-8890-3e49810ca94f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:56.668746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbf57e-c0aa-4d57-8890-3e49810ca94f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:56.668746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbf57e-c0aa-4d57-8890-3e49810ca94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:56.668746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffbf57e-c0aa-4d57-8890-3e49810ca94f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:56.668746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 552/1109] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:06:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:56.674304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c987e6-bf59-4e2d-9180-dc89932a2ddd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:56.703931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:56.808373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c987e6-bf59-4e2d-9180-dc89932a2ddd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:56.843151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b8817-5a10-46aa-8519-fb842cfb6eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:56.843970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8817-5a10-46aa-8519-fb842cfb6eb3 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:56.844249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8817-5a10-46aa-8519-fb842cfb6eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:56.844519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b8817-5a10-46aa-8519-fb842cfb6eb3 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:56.845224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b8817-5a10-46aa-8519-fb842cfb6eb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:56.846708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b8817-5a10-46aa-8519-fb842cfb6eb3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:56.870925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 558/1110] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:06:56 2022] POST /identity/v3/users => generated 286 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:06:59.036741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcce85-f771-4a66-acd1-c6f15b812572 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:59.037302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcce85-f771-4a66-acd1-c6f15b812572 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:59.037397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcce85-f771-4a66-acd1-c6f15b812572 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:59.038995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 553/1111] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:06:59.044922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504803f5-c109-428b-a662-88fd0edfec37 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:59.045417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504803f5-c109-428b-a662-88fd0edfec37 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:59.045533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504803f5-c109-428b-a662-88fd0edfec37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:59.063676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-504803f5-c109-428b-a662-88fd0edfec37 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:06:59.064920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-504803f5-c109-428b-a662-88fd0edfec37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:06:59.138858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 559/1112] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:06:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:06:59.162824 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcce85-f771-4a66-acd1-c6f15b812572 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:59.263848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:59.278898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcce85-f771-4a66-acd1-c6f15b812572 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:59.314621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85f1e9-01be-4cc5-ba9f-8d00fc65d422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:59.315358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85f1e9-01be-4cc5-ba9f-8d00fc65d422 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:59.315642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85f1e9-01be-4cc5-ba9f-8d00fc65d422 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:59.315910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85f1e9-01be-4cc5-ba9f-8d00fc65d422 None admin] PATH_INFO: `/v3/roles/89d5f9aaa6284371b4d5205fea92cba1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:59.317391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85f1e9-01be-4cc5-ba9f-8d00fc65d422 None admin] RBAC: Authorizing `identity:get_role(role_id=89d5f9aaa6284371b4d5205fea92cba1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:59.318955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85f1e9-01be-4cc5-ba9f-8d00fc65d422 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:59.320831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 554/1113] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:06:59 2022] GET /identity/v3/roles/89d5f9aaa6284371b4d5205fea92cba1 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:59.327084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504803f5-c109-428b-a662-88fd0edfec37 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:59.336768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:59.485319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504803f5-c109-428b-a662-88fd0edfec37 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:59.524243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ebf1-0a8c-4395-8af7-0d572ced8be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:59.525123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ebf1-0a8c-4395-8af7-0d572ced8be5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:59.525404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ebf1-0a8c-4395-8af7-0d572ced8be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:59.525735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2ebf1-0a8c-4395-8af7-0d572ced8be5 None admin] PATH_INFO: `/v3/users/1dd8e19f9a964fa08f100c5c2769c2b8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:59.540427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2ebf1-0a8c-4395-8af7-0d572ced8be5 None admin] RBAC: Authorizing `identity:get_user(user_id=1dd8e19f9a964fa08f100c5c2769c2b8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:59.541953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2ebf1-0a8c-4395-8af7-0d572ced8be5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:59.543707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 560/1114] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:06:59 2022] GET /identity/v3/users/1dd8e19f9a964fa08f100c5c2769c2b8 => generated 286 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:06:59.550601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85f1e9-01be-4cc5-ba9f-8d00fc65d422 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:59.666426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85f1e9-01be-4cc5-ba9f-8d00fc65d422 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:06:59.703849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7959e4-2d42-4967-ab12-bcf844a09213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:06:59.704503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7959e4-2d42-4967-ab12-bcf844a09213 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:06:59.704787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7959e4-2d42-4967-ab12-bcf844a09213 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:06:59.705061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7959e4-2d42-4967-ab12-bcf844a09213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:06:59.732100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:06:59.779960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7959e4-2d42-4967-ab12-bcf844a09213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:06:59.781439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7959e4-2d42-4967-ab12-bcf844a09213 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:06:59.890806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 555/1115] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:06:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:06:59.896978 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ebf1-0a8c-4395-8af7-0d572ced8be5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:06:59.971255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:00.014545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2ebf1-0a8c-4395-8af7-0d572ced8be5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:00.053374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380bf22-f46a-48ce-8a96-82581f2f2bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:00.054985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380bf22-f46a-48ce-8a96-82581f2f2bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:00.055270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380bf22-f46a-48ce-8a96-82581f2f2bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:00.055554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380bf22-f46a-48ce-8a96-82581f2f2bd2 None admin] PATH_INFO: `/v3/projects/6b0eeb2626d7481fba333867ef047f88` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:00.057071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380bf22-f46a-48ce-8a96-82581f2f2bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=6b0eeb2626d7481fba333867ef047f88)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:00.058587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380bf22-f46a-48ce-8a96-82581f2f2bd2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:00.060650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 561/1116] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:06:59 2022] GET /identity/v3/projects/6b0eeb2626d7481fba333867ef047f88 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:00.069433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7959e4-2d42-4967-ab12-bcf844a09213 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:00.139033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:00.189088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7959e4-2d42-4967-ab12-bcf844a09213 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:00.225502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13815b62-9717-43e6-a815-0c56d3ca8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:00.226619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13815b62-9717-43e6-a815-0c56d3ca8906 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:00.226619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13815b62-9717-43e6-a815-0c56d3ca8906 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:00.226854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13815b62-9717-43e6-a815-0c56d3ca8906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:00.227670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13815b62-9717-43e6-a815-0c56d3ca8906 None admin] RBAC: Adding query filter params (role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=6b0eeb2626d7481fba333867ef047f88, user.id=1dd8e19f9a964fa08f100c5c2769c2b8) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:00.227991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13815b62-9717-43e6-a815-0c56d3ca8906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:00.229448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13815b62-9717-43e6-a815-0c56d3ca8906 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:00.235430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 556/1117] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:07:00 2022] GET /identity/v3/role_assignments?user.id=1dd8e19f9a964fa08f100c5c2769c2b8&scope.project.id=6b0eeb2626d7481fba333867ef047f88&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:07:02.247350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3f17-69a6-4e67-94b3-c3a0cd1e145c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:02.247942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3f17-69a6-4e67-94b3-c3a0cd1e145c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:02.247942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3f17-69a6-4e67-94b3-c3a0cd1e145c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:02.249299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 562/1118] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:02.255862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337c237-f825-4b42-bec5-5459ff0614cb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:02.256638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337c237-f825-4b42-bec5-5459ff0614cb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:02.257409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337c237-f825-4b42-bec5-5459ff0614cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:02.280925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7337c237-f825-4b42-bec5-5459ff0614cb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:02.282168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7337c237-f825-4b42-bec5-5459ff0614cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:02.355367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 557/1119] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:02.383263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb3f17-69a6-4e67-94b3-c3a0cd1e145c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:02.499694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb3f17-69a6-4e67-94b3-c3a0cd1e145c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:02.533105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:02.539726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f68d2-3dd3-4c0d-b74d-9c03f520b7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:02.540569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f68d2-3dd3-4c0d-b74d-9c03f520b7ae None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:02.540876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f68d2-3dd3-4c0d-b74d-9c03f520b7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:02.541151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f68d2-3dd3-4c0d-b74d-9c03f520b7ae None admin] PATH_INFO: `/v3/roles/89d5f9aaa6284371b4d5205fea92cba1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:02.542941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f68d2-3dd3-4c0d-b74d-9c03f520b7ae None admin] RBAC: Authorizing `identity:get_role(role_id=89d5f9aaa6284371b4d5205fea92cba1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:02.544335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f68d2-3dd3-4c0d-b74d-9c03f520b7ae None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:02.545736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 563/1120] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:02 2022] GET /identity/v3/roles/89d5f9aaa6284371b4d5205fea92cba1 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:02.552577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337c237-f825-4b42-bec5-5459ff0614cb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:02.616089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:02.671177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337c237-f825-4b42-bec5-5459ff0614cb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:02.708723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6cf8-faf1-40d1-b038-df6619a5d164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:02.709608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6cf8-faf1-40d1-b038-df6619a5d164 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:02.709927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6cf8-faf1-40d1-b038-df6619a5d164 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:02.710298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6cf8-faf1-40d1-b038-df6619a5d164 None admin] PATH_INFO: `/v3/users/1dd8e19f9a964fa08f100c5c2769c2b8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:02.711659 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6cf8-faf1-40d1-b038-df6619a5d164 None admin] RBAC: Authorizing `identity:get_user(user_id=1dd8e19f9a964fa08f100c5c2769c2b8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:02.713108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6cf8-faf1-40d1-b038-df6619a5d164 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:02.714782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 558/1121] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:02 2022] GET /identity/v3/users/1dd8e19f9a964fa08f100c5c2769c2b8 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:02.721710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f68d2-3dd3-4c0d-b74d-9c03f520b7ae None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:02.833113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f68d2-3dd3-4c0d-b74d-9c03f520b7ae None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:02.872688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7215f7-137d-4bc2-a13a-1ac25dd9cfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:02.873540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7215f7-137d-4bc2-a13a-1ac25dd9cfef None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:02.873851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7215f7-137d-4bc2-a13a-1ac25dd9cfef None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:02.874130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7215f7-137d-4bc2-a13a-1ac25dd9cfef None admin] PATH_INFO: `/v3/projects/6b0eeb2626d7481fba333867ef047f88` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:02.875435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7215f7-137d-4bc2-a13a-1ac25dd9cfef None admin] RBAC: Authorizing `identity:get_project(project_id=6b0eeb2626d7481fba333867ef047f88)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:02.877164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7215f7-137d-4bc2-a13a-1ac25dd9cfef None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:02.878933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 564/1122] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:07:02 2022] GET /identity/v3/projects/6b0eeb2626d7481fba333867ef047f88 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:02.885562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6cf8-faf1-40d1-b038-df6619a5d164 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:02.996292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6cf8-faf1-40d1-b038-df6619a5d164 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:03.024372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:03.043607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe780b-7a90-4b9f-991d-59b7cff987c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:03.044510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe780b-7a90-4b9f-991d-59b7cff987c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:03.044796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe780b-7a90-4b9f-991d-59b7cff987c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:03.045074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe780b-7a90-4b9f-991d-59b7cff987c3 None admin] PATH_INFO: `/v3/projects/6b0eeb2626d7481fba333867ef047f88/users/1dd8e19f9a964fa08f100c5c2769c2b8/roles/89d5f9aaa6284371b4d5205fea92cba1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:03.047519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe780b-7a90-4b9f-991d-59b7cff987c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b0eeb2626d7481fba333867ef047f88, user_id=1dd8e19f9a964fa08f100c5c2769c2b8, role_id=89d5f9aaa6284371b4d5205fea92cba1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:03.048981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe780b-7a90-4b9f-991d-59b7cff987c3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:03.065647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 559/1123] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:07:02 2022] PUT /identity/v3/projects/6b0eeb2626d7481fba333867ef047f88/users/1dd8e19f9a964fa08f100c5c2769c2b8/roles/89d5f9aaa6284371b4d5205fea92cba1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:07:05.055349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7ef29-a492-4c7e-a66a-cae13ece11cb None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:05.055882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7ef29-a492-4c7e-a66a-cae13ece11cb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:05.055882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7ef29-a492-4c7e-a66a-cae13ece11cb None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:05.057209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 565/1124] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:05.063668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013b034-f665-4f45-a08b-ea0735354626 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:05.064515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013b034-f665-4f45-a08b-ea0735354626 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:05.064515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013b034-f665-4f45-a08b-ea0735354626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:05.085593 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2013b034-f665-4f45-a08b-ea0735354626 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:05.086837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2013b034-f665-4f45-a08b-ea0735354626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:05.163917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 560/1125] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:05.187637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7ef29-a492-4c7e-a66a-cae13ece11cb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:05.264865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:05.300673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7ef29-a492-4c7e-a66a-cae13ece11cb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:05.340526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50145336-53dd-4b2a-9788-162e370b4ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:05.341294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50145336-53dd-4b2a-9788-162e370b4ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:05.341570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50145336-53dd-4b2a-9788-162e370b4ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:05.341873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50145336-53dd-4b2a-9788-162e370b4ce8 None admin] PATH_INFO: `/v3/roles/89d5f9aaa6284371b4d5205fea92cba1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:05.343453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50145336-53dd-4b2a-9788-162e370b4ce8 None admin] RBAC: Authorizing `identity:get_role(role_id=89d5f9aaa6284371b4d5205fea92cba1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:05.344827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50145336-53dd-4b2a-9788-162e370b4ce8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:05.346251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 566/1126] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:05 2022] GET /identity/v3/roles/89d5f9aaa6284371b4d5205fea92cba1 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:05.352595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013b034-f665-4f45-a08b-ea0735354626 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:05.463806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013b034-f665-4f45-a08b-ea0735354626 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:05.474865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:05.505322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90df85-e4c7-4733-ad6d-dd4a92de5449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:05.506237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90df85-e4c7-4733-ad6d-dd4a92de5449 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:05.506517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90df85-e4c7-4733-ad6d-dd4a92de5449 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:05.506790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90df85-e4c7-4733-ad6d-dd4a92de5449 None admin] PATH_INFO: `/v3/users/1dd8e19f9a964fa08f100c5c2769c2b8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:05.508513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90df85-e4c7-4733-ad6d-dd4a92de5449 None admin] RBAC: Authorizing `identity:get_user(user_id=1dd8e19f9a964fa08f100c5c2769c2b8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:05.510003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef90df85-e4c7-4733-ad6d-dd4a92de5449 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:05.511651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 561/1127] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:05 2022] GET /identity/v3/users/1dd8e19f9a964fa08f100c5c2769c2b8 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:05.517930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50145336-53dd-4b2a-9788-162e370b4ce8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:05.623437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50145336-53dd-4b2a-9788-162e370b4ce8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:05.660484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4861ff4-9efa-41f5-b6c2-beb96b2eca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:05.661135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4861ff4-9efa-41f5-b6c2-beb96b2eca09 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:05.661414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4861ff4-9efa-41f5-b6c2-beb96b2eca09 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:05.661715 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4861ff4-9efa-41f5-b6c2-beb96b2eca09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:05.665450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:05.734792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4861ff4-9efa-41f5-b6c2-beb96b2eca09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:05.736241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4861ff4-9efa-41f5-b6c2-beb96b2eca09 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:05.855008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 567/1128] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:07:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:07:05.861478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90df85-e4c7-4733-ad6d-dd4a92de5449 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:05.987808 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90df85-e4c7-4733-ad6d-dd4a92de5449 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:06.027610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb2896-e9fb-45eb-8a3f-cee0cd123276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:06.028850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2896-e9fb-45eb-8a3f-cee0cd123276 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:06.029222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2896-e9fb-45eb-8a3f-cee0cd123276 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:06.029590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2896-e9fb-45eb-8a3f-cee0cd123276 None admin] PATH_INFO: `/v3/projects/6b0eeb2626d7481fba333867ef047f88` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:06.031739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb2896-e9fb-45eb-8a3f-cee0cd123276 None admin] RBAC: Authorizing `identity:get_project(project_id=6b0eeb2626d7481fba333867ef047f88)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:06.034052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb2896-e9fb-45eb-8a3f-cee0cd123276 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:06.036718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 562/1129] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:07:05 2022] GET /identity/v3/projects/6b0eeb2626d7481fba333867ef047f88 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:06.046554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4861ff4-9efa-41f5-b6c2-beb96b2eca09 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:06.088520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:06.158591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4861ff4-9efa-41f5-b6c2-beb96b2eca09 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:06.197820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3148adf-5216-44e6-be68-6973e21acb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:06.198594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148adf-5216-44e6-be68-6973e21acb1b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:06.198988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148adf-5216-44e6-be68-6973e21acb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:06.199353 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3148adf-5216-44e6-be68-6973e21acb1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:06.200271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3148adf-5216-44e6-be68-6973e21acb1b None admin] RBAC: Adding query filter params (role.id=89d5f9aaa6284371b4d5205fea92cba1, scope.project.id=6b0eeb2626d7481fba333867ef047f88, user.id=1dd8e19f9a964fa08f100c5c2769c2b8) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:06.200679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3148adf-5216-44e6-be68-6973e21acb1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:06.202333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3148adf-5216-44e6-be68-6973e21acb1b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:06.209402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 568/1130] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:07:06 2022] GET /identity/v3/role_assignments?user.id=1dd8e19f9a964fa08f100c5c2769c2b8&scope.project.id=6b0eeb2626d7481fba333867ef047f88&role.id=89d5f9aaa6284371b4d5205fea92cba1 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:08.295334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f267a-cc01-4e1b-b444-c332cd055315 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:08.296212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f267a-cc01-4e1b-b444-c332cd055315 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:08.296212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f267a-cc01-4e1b-b444-c332cd055315 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:08.298502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 563/1131] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:08.307184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b6824-a20e-415e-b2d4-257cfa0a084f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:08.307867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b6824-a20e-415e-b2d4-257cfa0a084f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:08.308291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b6824-a20e-415e-b2d4-257cfa0a084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:08.335112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5a5b6824-a20e-415e-b2d4-257cfa0a084f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:08.336538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5a5b6824-a20e-415e-b2d4-257cfa0a084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:08.413972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 569/1132] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:08.437829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f267a-cc01-4e1b-b444-c332cd055315 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:08.495239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:08.563733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f267a-cc01-4e1b-b444-c332cd055315 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:08.599884 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2161403-c280-4e28-882c-45e3e8e1ee1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:08.600748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161403-c280-4e28-882c-45e3e8e1ee1c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:08.601052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161403-c280-4e28-882c-45e3e8e1ee1c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:08.601412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161403-c280-4e28-882c-45e3e8e1ee1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:08.602691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2161403-c280-4e28-882c-45e3e8e1ee1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:08.604139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2161403-c280-4e28-882c-45e3e8e1ee1c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:08.605823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 564/1133] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:07:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:08.612155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b6824-a20e-415e-b2d4-257cfa0a084f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:08.644307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:08.720886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b6824-a20e-415e-b2d4-257cfa0a084f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:08.760682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375585d-efeb-47d8-99a0-811e5408ac2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:08.761344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375585d-efeb-47d8-99a0-811e5408ac2b None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:08.761628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375585d-efeb-47d8-99a0-811e5408ac2b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:08.761932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375585d-efeb-47d8-99a0-811e5408ac2b None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:08.762643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375585d-efeb-47d8-99a0-811e5408ac2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:08.764109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375585d-efeb-47d8-99a0-811e5408ac2b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:08.793340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 570/1134] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:07:08 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:07:11.035668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5f672-cd3d-4a64-a7d8-b9f0e8362b8e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:11.036208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5f672-cd3d-4a64-a7d8-b9f0e8362b8e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:11.036208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5f672-cd3d-4a64-a7d8-b9f0e8362b8e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:11.038312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 565/1135] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:11.046252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae411d1-e8c8-4e19-b836-28686cf43cd7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:11.046736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae411d1-e8c8-4e19-b836-28686cf43cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:11.046848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae411d1-e8c8-4e19-b836-28686cf43cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:11.070678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-dae411d1-e8c8-4e19-b836-28686cf43cd7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:11.071897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dae411d1-e8c8-4e19-b836-28686cf43cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:11.149536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 571/1136] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:11.174806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5f672-cd3d-4a64-a7d8-b9f0e8362b8e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:11.284500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5f672-cd3d-4a64-a7d8-b9f0e8362b8e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:11.298194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:11.319663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776259b-5684-4623-beb0-f212da3b7450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:11.320486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776259b-5684-4623-beb0-f212da3b7450 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:11.320760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776259b-5684-4623-beb0-f212da3b7450 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:11.321032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776259b-5684-4623-beb0-f212da3b7450 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:11.322263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776259b-5684-4623-beb0-f212da3b7450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:11.323697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776259b-5684-4623-beb0-f212da3b7450 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:11.325302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 566/1137] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:07:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:11.331475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae411d1-e8c8-4e19-b836-28686cf43cd7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:11.347814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:11.440811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae411d1-e8c8-4e19-b836-28686cf43cd7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:11.476456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba3f76-fb61-4ff6-b2e9-afd9ad8aa845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:11.477129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba3f76-fb61-4ff6-b2e9-afd9ad8aa845 None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:11.477417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba3f76-fb61-4ff6-b2e9-afd9ad8aa845 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:11.477806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba3f76-fb61-4ff6-b2e9-afd9ad8aa845 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:11.478515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba3f76-fb61-4ff6-b2e9-afd9ad8aa845 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:11.479967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba3f76-fb61-4ff6-b2e9-afd9ad8aa845 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:11.502059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 572/1138] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:07:11 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:07:13.581088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f481-564a-4875-a7cd-afc15539e544 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:13.582198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f481-564a-4875-a7cd-afc15539e544 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:13.582198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131f481-564a-4875-a7cd-afc15539e544 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:13.584280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 567/1139] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:13.590291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5ee4e-8892-4b69-985d-f99cfca54314 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:13.590769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5ee4e-8892-4b69-985d-f99cfca54314 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:13.590914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5ee4e-8892-4b69-985d-f99cfca54314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:13.608453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bad5ee4e-8892-4b69-985d-f99cfca54314 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:13.609632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bad5ee4e-8892-4b69-985d-f99cfca54314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:13.679128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:13.688184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 573/1140] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:13.719118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f481-564a-4875-a7cd-afc15539e544 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:13.840675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131f481-564a-4875-a7cd-afc15539e544 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:13.879271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d911d-023f-4300-bc54-ea81f664538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:13.880067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d911d-023f-4300-bc54-ea81f664538e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:13.880379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d911d-023f-4300-bc54-ea81f664538e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:13.880656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d911d-023f-4300-bc54-ea81f664538e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:13.891103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d911d-023f-4300-bc54-ea81f664538e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-f30d911d-023f-4300-bc54-ea81f664538e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:07:13.898928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:07:13.899969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:07:13.901107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 568/1141] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:07:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:07:13.906457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5ee4e-8892-4b69-985d-f99cfca54314 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:14.016793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5ee4e-8892-4b69-985d-f99cfca54314 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:14.062996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:14.063945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:14.064325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:14.064700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:14.065876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:14.066285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:14.068467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:14.077315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 574/1142] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:07:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:14.086125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d911d-023f-4300-bc54-ea81f664538e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:14.141700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:14.208640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d911d-023f-4300-bc54-ea81f664538e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:14.246600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980810f-d750-4e42-8ee0-7d5f6ce89764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:14.247473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980810f-d750-4e42-8ee0-7d5f6ce89764 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:14.247754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980810f-d750-4e42-8ee0-7d5f6ce89764 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:14.248092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980810f-d750-4e42-8ee0-7d5f6ce89764 None admin] PATH_INFO: `/v3/users/42a728272ee6498ba3fdc0f73bbcb74d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:14.262599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980810f-d750-4e42-8ee0-7d5f6ce89764 None admin] RBAC: Authorizing `identity:get_user(user_id=42a728272ee6498ba3fdc0f73bbcb74d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:14.264056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980810f-d750-4e42-8ee0-7d5f6ce89764 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:14.265805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 569/1143] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:14 2022] GET /identity/v3/users/42a728272ee6498ba3fdc0f73bbcb74d => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:14.274824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:14.347614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:14.395877 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43e536-0037-4808-accd-dff0e5c37d00 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:14.434929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c595d-7c1f-46ae-ac02-09efcc48f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:14.435575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c595d-7c1f-46ae-ac02-09efcc48f351 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:14.435874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c595d-7c1f-46ae-ac02-09efcc48f351 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:14.436112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c595d-7c1f-46ae-ac02-09efcc48f351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:14.509930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c595d-7c1f-46ae-ac02-09efcc48f351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:14.511398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c595d-7c1f-46ae-ac02-09efcc48f351 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:14.572282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:14.625389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 575/1144] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:07:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:07:14.631561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980810f-d750-4e42-8ee0-7d5f6ce89764 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:14.743783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:14.747873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980810f-d750-4e42-8ee0-7d5f6ce89764 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:14.782552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480aa2c-6a01-4b3c-b48c-7bab9516d1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:14.783415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480aa2c-6a01-4b3c-b48c-7bab9516d1ce None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:14.783702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480aa2c-6a01-4b3c-b48c-7bab9516d1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:14.783978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480aa2c-6a01-4b3c-b48c-7bab9516d1ce None admin] PATH_INFO: `/v3/projects/02d922f485c64eca9d3e3dd8e0586d56` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:14.785179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480aa2c-6a01-4b3c-b48c-7bab9516d1ce None admin] RBAC: Authorizing `identity:get_project(project_id=02d922f485c64eca9d3e3dd8e0586d56)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:14.786667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480aa2c-6a01-4b3c-b48c-7bab9516d1ce None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:14.788431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 570/1145] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:07:14 2022] GET /identity/v3/projects/02d922f485c64eca9d3e3dd8e0586d56 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:14.797612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c595d-7c1f-46ae-ac02-09efcc48f351 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:14.905739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c595d-7c1f-46ae-ac02-09efcc48f351 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:14.942416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ed7b9-2ae9-4df8-b826-9716619fd568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:14.943069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ed7b9-2ae9-4df8-b826-9716619fd568 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:14.943352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ed7b9-2ae9-4df8-b826-9716619fd568 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:14.943630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ed7b9-2ae9-4df8-b826-9716619fd568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:14.944544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ed7b9-2ae9-4df8-b826-9716619fd568 None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=02d922f485c64eca9d3e3dd8e0586d56, user.id=42a728272ee6498ba3fdc0f73bbcb74d) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:14.944848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ed7b9-2ae9-4df8-b826-9716619fd568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:14.946342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ed7b9-2ae9-4df8-b826-9716619fd568 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:14.952048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 576/1146] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:07:14 2022] GET /identity/v3/role_assignments?user.id=42a728272ee6498ba3fdc0f73bbcb74d&scope.project.id=02d922f485c64eca9d3e3dd8e0586d56&role.id=5c672a1b2979444383c587a215adeb35 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:07:16.948204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353238-8ca9-432e-a102-7ee9a6bdceb0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:16.949027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353238-8ca9-432e-a102-7ee9a6bdceb0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:16.949027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353238-8ca9-432e-a102-7ee9a6bdceb0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:16.951221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 571/1147] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:16.956702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129b492-cd4f-4241-b475-e43bf60aaac3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:16.957133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129b492-cd4f-4241-b475-e43bf60aaac3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:16.957240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7129b492-cd4f-4241-b475-e43bf60aaac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:16.975917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7129b492-cd4f-4241-b475-e43bf60aaac3 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:16.977297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7129b492-cd4f-4241-b475-e43bf60aaac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:16.997008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:17.058977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 577/1148] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:17.081618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22353238-8ca9-432e-a102-7ee9a6bdceb0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:17.212738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22353238-8ca9-432e-a102-7ee9a6bdceb0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:17.250840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933ab0b-c055-4f40-a75e-eae6275d88e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:17.251641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933ab0b-c055-4f40-a75e-eae6275d88e5 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:17.251927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933ab0b-c055-4f40-a75e-eae6275d88e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:17.252211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933ab0b-c055-4f40-a75e-eae6275d88e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:17.263748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933ab0b-c055-4f40-a75e-eae6275d88e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-8933ab0b-c055-4f40-a75e-eae6275d88e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:07:17.271380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:07:17.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:07:17.273864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 572/1149] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:07:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:07:17.279220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129b492-cd4f-4241-b475-e43bf60aaac3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:17.391658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7129b492-cd4f-4241-b475-e43bf60aaac3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:17.434107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:17.435051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:17.435408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:17.435763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:17.436855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:17.437279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:17.439316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:17.453306 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 578/1150] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:07:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:17.462895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933ab0b-c055-4f40-a75e-eae6275d88e5 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:17.502731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:17.581447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933ab0b-c055-4f40-a75e-eae6275d88e5 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:17.617251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2174bbe-314b-4c97-a9d0-ad7e13178772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:17.618465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2174bbe-314b-4c97-a9d0-ad7e13178772 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:17.618753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2174bbe-314b-4c97-a9d0-ad7e13178772 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:17.619021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2174bbe-314b-4c97-a9d0-ad7e13178772 None admin] PATH_INFO: `/v3/users/42a728272ee6498ba3fdc0f73bbcb74d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:17.620348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2174bbe-314b-4c97-a9d0-ad7e13178772 None admin] RBAC: Authorizing `identity:get_user(user_id=42a728272ee6498ba3fdc0f73bbcb74d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:17.621833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2174bbe-314b-4c97-a9d0-ad7e13178772 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:17.623491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 573/1151] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:17 2022] GET /identity/v3/users/42a728272ee6498ba3fdc0f73bbcb74d => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:17.629600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:17.655419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:17.746030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62536683-2741-4de8-8fe1-24498b1bf9c8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:17.781320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b24c9-8224-447d-8c75-14920be4408d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:17.782234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b24c9-8224-447d-8c75-14920be4408d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:17.782516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b24c9-8224-447d-8c75-14920be4408d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:17.782791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b24c9-8224-447d-8c75-14920be4408d None admin] PATH_INFO: `/v3/projects/02d922f485c64eca9d3e3dd8e0586d56` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:17.784010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b24c9-8224-447d-8c75-14920be4408d None admin] RBAC: Authorizing `identity:get_project(project_id=02d922f485c64eca9d3e3dd8e0586d56)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:17.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b24c9-8224-447d-8c75-14920be4408d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:17.787218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 579/1152] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:07:17 2022] GET /identity/v3/projects/02d922f485c64eca9d3e3dd8e0586d56 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:17.794169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2174bbe-314b-4c97-a9d0-ad7e13178772 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:17.909064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:17.916251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2174bbe-314b-4c97-a9d0-ad7e13178772 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:17.952094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f82097-b317-4a75-9507-7be79157d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:17.953010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82097-b317-4a75-9507-7be79157d843 None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:17.953295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82097-b317-4a75-9507-7be79157d843 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:17.953559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82097-b317-4a75-9507-7be79157d843 None admin] PATH_INFO: `/v3/projects/02d922f485c64eca9d3e3dd8e0586d56/users/42a728272ee6498ba3fdc0f73bbcb74d/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:17.955693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f82097-b317-4a75-9507-7be79157d843 None admin] RBAC: Authorizing `identity:create_grant(project_id=02d922f485c64eca9d3e3dd8e0586d56, user_id=42a728272ee6498ba3fdc0f73bbcb74d, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:17.957169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f82097-b317-4a75-9507-7be79157d843 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:17.972447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 574/1153] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:07:17 2022] PUT /identity/v3/projects/02d922f485c64eca9d3e3dd8e0586d56/users/42a728272ee6498ba3fdc0f73bbcb74d/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:07:19.981419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef08381-f01f-4e5d-b7d8-5cf7b39af15a None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:19.982059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef08381-f01f-4e5d-b7d8-5cf7b39af15a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:19.982059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef08381-f01f-4e5d-b7d8-5cf7b39af15a None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:19.983285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 580/1154] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:19.988650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc41d67-630b-4a8e-883b-35c778929055 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:19.989151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc41d67-630b-4a8e-883b-35c778929055 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:19.989267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc41d67-630b-4a8e-883b-35c778929055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:20.008085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7cc41d67-630b-4a8e-883b-35c778929055 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:20.009290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7cc41d67-630b-4a8e-883b-35c778929055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:20.080758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 575/1155] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:20.104880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef08381-f01f-4e5d-b7d8-5cf7b39af15a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:20.210796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:20.230149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef08381-f01f-4e5d-b7d8-5cf7b39af15a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:20.269774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a21168-aaf6-410e-bcb4-3bd9cf741d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:20.270505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21168-aaf6-410e-bcb4-3bd9cf741d22 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:20.270786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21168-aaf6-410e-bcb4-3bd9cf741d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:20.271060 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a21168-aaf6-410e-bcb4-3bd9cf741d22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:20.283923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a21168-aaf6-410e-bcb4-3bd9cf741d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-b8a21168-aaf6-410e-bcb4-3bd9cf741d22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:07:20.291709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:07:20.292885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:07:20.294292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 581/1156] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:07:20.300023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc41d67-630b-4a8e-883b-35c778929055 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:20.393695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:20.412294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc41d67-630b-4a8e-883b-35c778929055 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:20.453269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:20.454230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:20.454602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:20.454955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:20.456110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:20.456526 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:20.458686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:20.467693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 576/1157] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:20.473954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a21168-aaf6-410e-bcb4-3bd9cf741d22 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:20.589827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a21168-aaf6-410e-bcb4-3bd9cf741d22 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:20.632063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d766f8e-cbe9-4b9d-afa2-674ba56749fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:20.633078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d766f8e-cbe9-4b9d-afa2-674ba56749fa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:20.633359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d766f8e-cbe9-4b9d-afa2-674ba56749fa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:20.633629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d766f8e-cbe9-4b9d-afa2-674ba56749fa None admin] PATH_INFO: `/v3/users/42a728272ee6498ba3fdc0f73bbcb74d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:20.637339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d766f8e-cbe9-4b9d-afa2-674ba56749fa None admin] RBAC: Authorizing `identity:get_user(user_id=42a728272ee6498ba3fdc0f73bbcb74d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:20.638824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d766f8e-cbe9-4b9d-afa2-674ba56749fa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:20.640487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 582/1158] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:20 2022] GET /identity/v3/users/42a728272ee6498ba3fdc0f73bbcb74d => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:20.646675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:20.757100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b60f0ff-4af4-4b04-82bc-847a4f7b610d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:20.793187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26b5c2-1e28-4cca-a287-d110cc94dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:20.793848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26b5c2-1e28-4cca-a287-d110cc94dac0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:20.794136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26b5c2-1e28-4cca-a287-d110cc94dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:20.794407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26b5c2-1e28-4cca-a287-d110cc94dac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:20.809421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:20.867299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26b5c2-1e28-4cca-a287-d110cc94dac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:20.869195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26b5c2-1e28-4cca-a287-d110cc94dac0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:20.981946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 577/1159] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:07:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:07:20.991349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d766f8e-cbe9-4b9d-afa2-674ba56749fa None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:21.008317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:21.105176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d766f8e-cbe9-4b9d-afa2-674ba56749fa None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:21.146829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebaf71-dc2f-4d4a-978d-c7b208b8989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:21.148267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebaf71-dc2f-4d4a-978d-c7b208b8989f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:21.148587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebaf71-dc2f-4d4a-978d-c7b208b8989f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:21.148831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebaf71-dc2f-4d4a-978d-c7b208b8989f None admin] PATH_INFO: `/v3/projects/02d922f485c64eca9d3e3dd8e0586d56` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:21.150041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebaf71-dc2f-4d4a-978d-c7b208b8989f None admin] RBAC: Authorizing `identity:get_project(project_id=02d922f485c64eca9d3e3dd8e0586d56)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:21.151500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebaf71-dc2f-4d4a-978d-c7b208b8989f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:21.153220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 583/1160] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:07:20 2022] GET /identity/v3/projects/02d922f485c64eca9d3e3dd8e0586d56 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:21.159849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26b5c2-1e28-4cca-a287-d110cc94dac0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:21.212509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:21.279071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26b5c2-1e28-4cca-a287-d110cc94dac0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:21.327922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67719f-5ebc-44b3-a858-0155c42f681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:21.328603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67719f-5ebc-44b3-a858-0155c42f681a None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:21.328886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67719f-5ebc-44b3-a858-0155c42f681a None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:21.329158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67719f-5ebc-44b3-a858-0155c42f681a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:21.330031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67719f-5ebc-44b3-a858-0155c42f681a None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=02d922f485c64eca9d3e3dd8e0586d56, user.id=42a728272ee6498ba3fdc0f73bbcb74d) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:21.330344 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67719f-5ebc-44b3-a858-0155c42f681a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:21.331783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67719f-5ebc-44b3-a858-0155c42f681a None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:21.338317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 578/1161] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:07:21 2022] GET /identity/v3/role_assignments?user.id=42a728272ee6498ba3fdc0f73bbcb74d&scope.project.id=02d922f485c64eca9d3e3dd8e0586d56&role.id=5c672a1b2979444383c587a215adeb35 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:23.368919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87585791-968a-46f7-97b7-09ba6be3169c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:23.369918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87585791-968a-46f7-97b7-09ba6be3169c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:23.369918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87585791-968a-46f7-97b7-09ba6be3169c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:23.372074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 584/1162] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:23.378483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a8da-c50e-40da-90bb-ea4bbff9e78d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:23.378964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a8da-c50e-40da-90bb-ea4bbff9e78d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:23.379066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377a8da-c50e-40da-90bb-ea4bbff9e78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:23.403767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6377a8da-c50e-40da-90bb-ea4bbff9e78d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:23.404955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6377a8da-c50e-40da-90bb-ea4bbff9e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:23.479440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 579/1163] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:23.502450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87585791-968a-46f7-97b7-09ba6be3169c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:23.594066 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:23.627832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87585791-968a-46f7-97b7-09ba6be3169c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:23.663722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836ca17-d586-42a0-8a01-82b814e90857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:23.664377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836ca17-d586-42a0-8a01-82b814e90857 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:23.664658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836ca17-d586-42a0-8a01-82b814e90857 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:23.664932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836ca17-d586-42a0-8a01-82b814e90857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:23.734487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836ca17-d586-42a0-8a01-82b814e90857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:23.735954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836ca17-d586-42a0-8a01-82b814e90857 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:23.830358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:23.852005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 585/1164] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:07:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:07:23.858357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377a8da-c50e-40da-90bb-ea4bbff9e78d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:23.904177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:23.964729 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377a8da-c50e-40da-90bb-ea4bbff9e78d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:24.002967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cf5a3-9c9f-4313-8502-2ced412f30b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:24.003779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cf5a3-9c9f-4313-8502-2ced412f30b3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:24.004062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cf5a3-9c9f-4313-8502-2ced412f30b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:24.004334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cf5a3-9c9f-4313-8502-2ced412f30b3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:24.012420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cf5a3-9c9f-4313-8502-2ced412f30b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:24.013912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cf5a3-9c9f-4313-8502-2ced412f30b3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-103cf5a3-9c9f-4313-8502-2ced412f30b3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:07:24.029110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:07:24.030390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:07:24.031269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 580/1165] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:07:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:07:24.037302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836ca17-d586-42a0-8a01-82b814e90857 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:24.162396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836ca17-d586-42a0-8a01-82b814e90857 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:24.201556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec821c-1c45-434a-8b36-664226906e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:24.202249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec821c-1c45-434a-8b36-664226906e18 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:24.202518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec821c-1c45-434a-8b36-664226906e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:24.202790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec821c-1c45-434a-8b36-664226906e18 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:24.203490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec821c-1c45-434a-8b36-664226906e18 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:24.203802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec821c-1c45-434a-8b36-664226906e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:24.205175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec821c-1c45-434a-8b36-664226906e18 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:24.212800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 586/1166] 10.222.0.71 () {56 vars in 1115 bytes} [Mon Jul 18 14:07:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:26.238988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecac415-3256-43b7-ab06-9d438949534c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:26.239590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecac415-3256-43b7-ab06-9d438949534c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:26.239590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecac415-3256-43b7-ab06-9d438949534c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:26.241056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 581/1167] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:26.247970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce737-a001-4139-b3d7-c12b60041a45 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:26.248746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce737-a001-4139-b3d7-c12b60041a45 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:26.248917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce737-a001-4139-b3d7-c12b60041a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:26.276338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3d0ce737-a001-4139-b3d7-c12b60041a45 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:26.277534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3d0ce737-a001-4139-b3d7-c12b60041a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:26.322029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:26.360314 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 587/1168] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:26.384873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecac415-3256-43b7-ab06-9d438949534c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:26.498193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:26.502382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecac415-3256-43b7-ab06-9d438949534c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:26.542011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506ced2-4426-45f7-9032-fc2de2f9b9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:26.542672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ced2-4426-45f7-9032-fc2de2f9b9f4 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:26.542950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ced2-4426-45f7-9032-fc2de2f9b9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:26.543226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506ced2-4426-45f7-9032-fc2de2f9b9f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:26.543968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506ced2-4426-45f7-9032-fc2de2f9b9f4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:26.545348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506ced2-4426-45f7-9032-fc2de2f9b9f4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:26.567914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 582/1169] 10.222.0.71 () {60 vars in 1138 bytes} [Mon Jul 18 14:07:26 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:07:28.693317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2947-622c-4b1b-94d0-52544cdb1a15 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:28.693980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2947-622c-4b1b-94d0-52544cdb1a15 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:28.693980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a2947-622c-4b1b-94d0-52544cdb1a15 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:28.695614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 588/1170] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:28.701690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e6797-2695-4659-a872-ae26f353a6ec None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:28.702152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e6797-2695-4659-a872-ae26f353a6ec None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:28.702251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e6797-2695-4659-a872-ae26f353a6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:28.720220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-687e6797-2695-4659-a872-ae26f353a6ec None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:28.721412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-687e6797-2695-4659-a872-ae26f353a6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:28.795695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 583/1171] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:28.819681 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a2947-622c-4b1b-94d0-52544cdb1a15 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:28.939990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a2947-622c-4b1b-94d0-52544cdb1a15 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:28.975889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da3ce8-a55f-455b-8c83-b1f49de4382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:28.976751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da3ce8-a55f-455b-8c83-b1f49de4382c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:28.977035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da3ce8-a55f-455b-8c83-b1f49de4382c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:28.977313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da3ce8-a55f-455b-8c83-b1f49de4382c None admin] PATH_INFO: `/v3/domains/53c06e4b66e943899006184158289c21` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:28.988589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da3ce8-a55f-455b-8c83-b1f49de4382c None admin] RBAC: Authorizing `identity:get_domain(domain_id=53c06e4b66e943899006184158289c21)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:28.990101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da3ce8-a55f-455b-8c83-b1f49de4382c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:28.992176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 589/1172] 10.222.0.71 () {56 vars in 1183 bytes} [Mon Jul 18 14:07:28 2022] GET /identity/v3/domains/53c06e4b66e943899006184158289c21 => generated 267 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:28.998762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e6797-2695-4659-a872-ae26f353a6ec None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:29.108951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:29.120537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e6797-2695-4659-a872-ae26f353a6ec None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:29.156980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cae66-da84-471f-bef8-6d66fcb9b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:29.157673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cae66-da84-471f-bef8-6d66fcb9b3e3 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:29.157975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cae66-da84-471f-bef8-6d66fcb9b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:29.158257 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cae66-da84-471f-bef8-6d66fcb9b3e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:29.158979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cae66-da84-471f-bef8-6d66fcb9b3e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:29.160428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cae66-da84-471f-bef8-6d66fcb9b3e3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:29.189621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 584/1173] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:07:28 2022] POST /identity/v3/projects => generated 361 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:07:31.316224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb4629-7e00-49ea-9d58-87430f3eb324 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:31.316805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb4629-7e00-49ea-9d58-87430f3eb324 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:31.316805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb4629-7e00-49ea-9d58-87430f3eb324 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:31.318222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 590/1174] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:31.325075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702369-4830-47a7-9c97-a2fd18be480f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:31.325605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702369-4830-47a7-9c97-a2fd18be480f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:31.325605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702369-4830-47a7-9c97-a2fd18be480f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:31.346253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7a702369-4830-47a7-9c97-a2fd18be480f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:31.348133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7a702369-4830-47a7-9c97-a2fd18be480f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:31.423326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 585/1175] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:31.450662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb4629-7e00-49ea-9d58-87430f3eb324 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:31.494160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:31.573437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb4629-7e00-49ea-9d58-87430f3eb324 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:31.609143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536530a9-d62b-4e9c-aec7-614b04291a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:31.610038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536530a9-d62b-4e9c-aec7-614b04291a47 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:31.610311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536530a9-d62b-4e9c-aec7-614b04291a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:31.610588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536530a9-d62b-4e9c-aec7-614b04291a47 None admin] PATH_INFO: `/v3/domains/53c06e4b66e943899006184158289c21` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:31.611781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536530a9-d62b-4e9c-aec7-614b04291a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=53c06e4b66e943899006184158289c21)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:31.613240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536530a9-d62b-4e9c-aec7-614b04291a47 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:31.614926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 591/1176] 10.222.0.71 () {56 vars in 1183 bytes} [Mon Jul 18 14:07:31 2022] GET /identity/v3/domains/53c06e4b66e943899006184158289c21 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:31.624039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a702369-4830-47a7-9c97-a2fd18be480f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:31.699701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:31.730097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a702369-4830-47a7-9c97-a2fd18be480f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:31.764961 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9ab33-a7a2-4fc7-a162-58a7fb363134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:31.765632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9ab33-a7a2-4fc7-a162-58a7fb363134 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:31.765947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9ab33-a7a2-4fc7-a162-58a7fb363134 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:31.766225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9ab33-a7a2-4fc7-a162-58a7fb363134 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:31.766949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9ab33-a7a2-4fc7-a162-58a7fb363134 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:31.768396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9ab33-a7a2-4fc7-a162-58a7fb363134 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:31.790491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 586/1177] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:07:31 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:07:33.862200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896055a6-103e-4059-9ba4-fff3efe80c14 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:33.862758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896055a6-103e-4059-9ba4-fff3efe80c14 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:33.862758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896055a6-103e-4059-9ba4-fff3efe80c14 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:33.864124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 592/1178] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:07:33.871190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66892a-0e91-4894-99ae-bea241e655e6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:33.871882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66892a-0e91-4894-99ae-bea241e655e6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:33.872069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66892a-0e91-4894-99ae-bea241e655e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:33.899150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7e66892a-0e91-4894-99ae-bea241e655e6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:33.900262 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7e66892a-0e91-4894-99ae-bea241e655e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:33.969190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 587/1179] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:33.994520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896055a6-103e-4059-9ba4-fff3efe80c14 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:34.113034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896055a6-103e-4059-9ba4-fff3efe80c14 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:34.114594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:34.158401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede13565-17fb-49fd-8eb5-45f1cd0efcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:34.159069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede13565-17fb-49fd-8eb5-45f1cd0efcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:34.159397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede13565-17fb-49fd-8eb5-45f1cd0efcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:34.159668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede13565-17fb-49fd-8eb5-45f1cd0efcc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:34.170769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede13565-17fb-49fd-8eb5-45f1cd0efcc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-ede13565-17fb-49fd-8eb5-45f1cd0efcc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:07:34.178627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:07:34.180149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:34.181169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:07:34.184940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 593/1180] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:07:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:07:34.186556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66892a-0e91-4894-99ae-bea241e655e6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:34.247547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:34.300562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66892a-0e91-4894-99ae-bea241e655e6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:34.346318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:34.347016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:34.347294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:34.347568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:34.348317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:34.348623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:34.350357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:34.358858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 588/1181] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:07:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:34.366085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede13565-17fb-49fd-8eb5-45f1cd0efcc0 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:34.479132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede13565-17fb-49fd-8eb5-45f1cd0efcc0 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:34.516175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb89ce4-f8a0-4f30-9e77-e0f164b7589b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:34.517049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89ce4-f8a0-4f30-9e77-e0f164b7589b None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:34.517328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89ce4-f8a0-4f30-9e77-e0f164b7589b None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:34.517602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb89ce4-f8a0-4f30-9e77-e0f164b7589b None admin] PATH_INFO: `/v3/users/9670f23ecbc14c3a9dc602fcbc32a441` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:34.532563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb89ce4-f8a0-4f30-9e77-e0f164b7589b None admin] RBAC: Authorizing `identity:get_user(user_id=9670f23ecbc14c3a9dc602fcbc32a441)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:34.534069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb89ce4-f8a0-4f30-9e77-e0f164b7589b None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:34.535778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 594/1182] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:34 2022] GET /identity/v3/users/9670f23ecbc14c3a9dc602fcbc32a441 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:34.544012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:34.650332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12dedd-cbc8-4455-91b2-f8a20e3d7206 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:34.671010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:34.686280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6541f3d-bfee-4a38-8a05-78d9ef5a7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:34.686932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6541f3d-bfee-4a38-8a05-78d9ef5a7358 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:34.687213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6541f3d-bfee-4a38-8a05-78d9ef5a7358 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:34.687485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6541f3d-bfee-4a38-8a05-78d9ef5a7358 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:34.774385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6541f3d-bfee-4a38-8a05-78d9ef5a7358 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:34.775754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6541f3d-bfee-4a38-8a05-78d9ef5a7358 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:34.885070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 589/1183] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:07:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:07:34.891972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb89ce4-f8a0-4f30-9e77-e0f164b7589b None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:34.915829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:35.013348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb89ce4-f8a0-4f30-9e77-e0f164b7589b None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:35.048474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43428cb-7225-42e8-9be0-e462b8d2be44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:35.049239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43428cb-7225-42e8-9be0-e462b8d2be44 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:35.049517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43428cb-7225-42e8-9be0-e462b8d2be44 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:35.049811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43428cb-7225-42e8-9be0-e462b8d2be44 None admin] PATH_INFO: `/v3/projects/ad717b2ddd324dbd9f727cf512b99e6b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:35.051007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43428cb-7225-42e8-9be0-e462b8d2be44 None admin] RBAC: Authorizing `identity:get_project(project_id=ad717b2ddd324dbd9f727cf512b99e6b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:35.052457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43428cb-7225-42e8-9be0-e462b8d2be44 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:35.054574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 595/1184] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:07:34 2022] GET /identity/v3/projects/ad717b2ddd324dbd9f727cf512b99e6b => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:35.060942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6541f3d-bfee-4a38-8a05-78d9ef5a7358 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:35.084488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:35.176209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6541f3d-bfee-4a38-8a05-78d9ef5a7358 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:35.222960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caaebb-8e3f-49f3-bb42-bec7100e65b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:35.223623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caaebb-8e3f-49f3-bb42-bec7100e65b0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:35.223908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caaebb-8e3f-49f3-bb42-bec7100e65b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:35.224166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caaebb-8e3f-49f3-bb42-bec7100e65b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:35.225259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caaebb-8e3f-49f3-bb42-bec7100e65b0 None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=ad717b2ddd324dbd9f727cf512b99e6b, user.id=9670f23ecbc14c3a9dc602fcbc32a441) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:35.225699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caaebb-8e3f-49f3-bb42-bec7100e65b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:35.227660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caaebb-8e3f-49f3-bb42-bec7100e65b0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:35.236234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 590/1185] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:07:35 2022] GET /identity/v3/role_assignments?user.id=9670f23ecbc14c3a9dc602fcbc32a441&scope.project.id=ad717b2ddd324dbd9f727cf512b99e6b&role.id=5c672a1b2979444383c587a215adeb35 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:37.256899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ea75d-abb7-4f87-b7ce-45bba9239d03 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:37.257511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ea75d-abb7-4f87-b7ce-45bba9239d03 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:37.257511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ea75d-abb7-4f87-b7ce-45bba9239d03 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:37.258922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 596/1186] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:37.264765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2420dc-5ccb-4a67-a159-9c6d30f407d8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:37.265234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2420dc-5ccb-4a67-a159-9c6d30f407d8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:37.265349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2420dc-5ccb-4a67-a159-9c6d30f407d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:37.284197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ed2420dc-5ccb-4a67-a159-9c6d30f407d8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:37.285420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ed2420dc-5ccb-4a67-a159-9c6d30f407d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:37.363605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 591/1187] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:37.392486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ea75d-abb7-4f87-b7ce-45bba9239d03 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:37.493376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:37.515868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ea75d-abb7-4f87-b7ce-45bba9239d03 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:37.555594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e801584-8d57-4dcb-adf0-ffb8ffae2dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:37.556391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e801584-8d57-4dcb-adf0-ffb8ffae2dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:37.556674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e801584-8d57-4dcb-adf0-ffb8ffae2dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:37.556950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e801584-8d57-4dcb-adf0-ffb8ffae2dd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:37.568129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e801584-8d57-4dcb-adf0-ffb8ffae2dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-3e801584-8d57-4dcb-adf0-ffb8ffae2dd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:07:37.575789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:07:37.576915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:07:37.582202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 597/1188] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:07:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:07:37.584951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2420dc-5ccb-4a67-a159-9c6d30f407d8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:37.600528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:37.695050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2420dc-5ccb-4a67-a159-9c6d30f407d8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:37.738779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:37.739460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:37.739742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:37.740015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:37.740930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:37.741236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:37.743012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:37.751446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 592/1189] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:07:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:37.761527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e801584-8d57-4dcb-adf0-ffb8ffae2dd1 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:37.875953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e801584-8d57-4dcb-adf0-ffb8ffae2dd1 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:37.910776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a84e9b-92d6-4a26-abf0-c0a79b457ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:37.911645 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a84e9b-92d6-4a26-abf0-c0a79b457ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:37.911925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a84e9b-92d6-4a26-abf0-c0a79b457ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:37.912236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a84e9b-92d6-4a26-abf0-c0a79b457ef7 None admin] PATH_INFO: `/v3/users/9670f23ecbc14c3a9dc602fcbc32a441` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:37.913699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a84e9b-92d6-4a26-abf0-c0a79b457ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=9670f23ecbc14c3a9dc602fcbc32a441)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:37.915144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a84e9b-92d6-4a26-abf0-c0a79b457ef7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:37.916776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 598/1190] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:37 2022] GET /identity/v3/users/9670f23ecbc14c3a9dc602fcbc32a441 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:37.926340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:38.034449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:38.068260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849d070-d612-4d1d-a06a-a8a558eff08d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:38.116443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10800615-7e58-4cf0-b6cf-6db8bf216115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:38.117703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800615-7e58-4cf0-b6cf-6db8bf216115 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:38.118001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800615-7e58-4cf0-b6cf-6db8bf216115 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:38.118275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800615-7e58-4cf0-b6cf-6db8bf216115 None admin] PATH_INFO: `/v3/projects/ad717b2ddd324dbd9f727cf512b99e6b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:38.119554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10800615-7e58-4cf0-b6cf-6db8bf216115 None admin] RBAC: Authorizing `identity:get_project(project_id=ad717b2ddd324dbd9f727cf512b99e6b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:38.120998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10800615-7e58-4cf0-b6cf-6db8bf216115 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:38.122806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 593/1191] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:07:37 2022] GET /identity/v3/projects/ad717b2ddd324dbd9f727cf512b99e6b => generated 361 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:38.130172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a84e9b-92d6-4a26-abf0-c0a79b457ef7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:38.156207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:38.243520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a84e9b-92d6-4a26-abf0-c0a79b457ef7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:38.281195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb467171-ef66-46b6-b9b4-18075963bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:38.282145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb467171-ef66-46b6-b9b4-18075963bf16 None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:38.282427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb467171-ef66-46b6-b9b4-18075963bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:38.282698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb467171-ef66-46b6-b9b4-18075963bf16 None admin] PATH_INFO: `/v3/projects/ad717b2ddd324dbd9f727cf512b99e6b/users/9670f23ecbc14c3a9dc602fcbc32a441/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:38.284747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb467171-ef66-46b6-b9b4-18075963bf16 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad717b2ddd324dbd9f727cf512b99e6b, user_id=9670f23ecbc14c3a9dc602fcbc32a441, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:38.286249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb467171-ef66-46b6-b9b4-18075963bf16 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:38.303460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 599/1192] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:07:38 2022] PUT /identity/v3/projects/ad717b2ddd324dbd9f727cf512b99e6b/users/9670f23ecbc14c3a9dc602fcbc32a441/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:07:40.353124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474efd6-9f73-40ff-8d1c-7193f16b01cd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:40.353713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474efd6-9f73-40ff-8d1c-7193f16b01cd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:40.353713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474efd6-9f73-40ff-8d1c-7193f16b01cd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:40.355195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 594/1193] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:07:40.361116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffff0c-1bbf-4068-91a1-fbf81ddee8d8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:40.361679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffff0c-1bbf-4068-91a1-fbf81ddee8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:40.361679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcffff0c-1bbf-4068-91a1-fbf81ddee8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:40.379558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bcffff0c-1bbf-4068-91a1-fbf81ddee8d8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:07:40.380934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bcffff0c-1bbf-4068-91a1-fbf81ddee8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:07:40.461515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 600/1194] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:07:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:07:40.485917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474efd6-9f73-40ff-8d1c-7193f16b01cd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:40.592106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474efd6-9f73-40ff-8d1c-7193f16b01cd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:40.626571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd57d68-5ccd-4900-b22d-6f77b36d01da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:40.627297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd57d68-5ccd-4900-b22d-6f77b36d01da None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:40.627583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd57d68-5ccd-4900-b22d-6f77b36d01da None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:40.627861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd57d68-5ccd-4900-b22d-6f77b36d01da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:40.640336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd57d68-5ccd-4900-b22d-6f77b36d01da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4bd57d68-5ccd-4900-b22d-6f77b36d01da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:07:40.648106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:07:40.649292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:07:40.650426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 595/1195] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:07:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:07:40.655764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffff0c-1bbf-4068-91a1-fbf81ddee8d8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:40.665599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:40.778040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcffff0c-1bbf-4068-91a1-fbf81ddee8d8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:40.816375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:40.817343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:40.817627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:40.818047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:40.818967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:40.819278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:40.820689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:40.829735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 601/1196] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:07:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:40.835848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd57d68-5ccd-4900-b22d-6f77b36d01da None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:40.840015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:40.945205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd57d68-5ccd-4900-b22d-6f77b36d01da None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:40.986973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6bdb8-1062-48d5-b80c-531fc2fdf9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:40.987872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6bdb8-1062-48d5-b80c-531fc2fdf9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:40.988146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6bdb8-1062-48d5-b80c-531fc2fdf9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:40.988423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6bdb8-1062-48d5-b80c-531fc2fdf9c0 None admin] PATH_INFO: `/v3/users/9670f23ecbc14c3a9dc602fcbc32a441` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:40.989685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6bdb8-1062-48d5-b80c-531fc2fdf9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=9670f23ecbc14c3a9dc602fcbc32a441)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:40.991217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6bdb8-1062-48d5-b80c-531fc2fdf9c0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:40.993227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 596/1197] 10.222.0.71 () {56 vars in 1177 bytes} [Mon Jul 18 14:07:40 2022] GET /identity/v3/users/9670f23ecbc14c3a9dc602fcbc32a441 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:40.999270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:41.076172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:41.114365 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376de8c-3e0d-455b-8a36-030fc841f3d7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:41.151911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b531ee1-21af-4434-95e6-06ae46dcb90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:41.152553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b531ee1-21af-4434-95e6-06ae46dcb90e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:41.152833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b531ee1-21af-4434-95e6-06ae46dcb90e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:41.153116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b531ee1-21af-4434-95e6-06ae46dcb90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:41.229001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b531ee1-21af-4434-95e6-06ae46dcb90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:41.230420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b531ee1-21af-4434-95e6-06ae46dcb90e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:41.299578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:41.338233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 602/1198] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:07:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:07:41.344514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6bdb8-1062-48d5-b80c-531fc2fdf9c0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:41.419678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:41.453133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6bdb8-1062-48d5-b80c-531fc2fdf9c0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:41.491584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70177a-8639-4116-b332-fc008172ea61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:41.492689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70177a-8639-4116-b332-fc008172ea61 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:41.492979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70177a-8639-4116-b332-fc008172ea61 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:41.493260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70177a-8639-4116-b332-fc008172ea61 None admin] PATH_INFO: `/v3/projects/ad717b2ddd324dbd9f727cf512b99e6b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:41.494574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70177a-8639-4116-b332-fc008172ea61 None admin] RBAC: Authorizing `identity:get_project(project_id=ad717b2ddd324dbd9f727cf512b99e6b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:41.496014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70177a-8639-4116-b332-fc008172ea61 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:41.497790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 597/1199] 10.222.0.71 () {56 vars in 1186 bytes} [Mon Jul 18 14:07:41 2022] GET /identity/v3/projects/ad717b2ddd324dbd9f727cf512b99e6b => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:41.504829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b531ee1-21af-4434-95e6-06ae46dcb90e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:07:41.614892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b531ee1-21af-4434-95e6-06ae46dcb90e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:07:41.653990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754ed95-8985-4b27-8ea5-114b377274ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:07:41.654961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754ed95-8985-4b27-8ea5-114b377274ff None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:07:41.655388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754ed95-8985-4b27-8ea5-114b377274ff None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:07:41.655808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754ed95-8985-4b27-8ea5-114b377274ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:07:41.657150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754ed95-8985-4b27-8ea5-114b377274ff None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=ad717b2ddd324dbd9f727cf512b99e6b, user.id=9670f23ecbc14c3a9dc602fcbc32a441) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:07:41.657584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754ed95-8985-4b27-8ea5-114b377274ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:07:41.659382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754ed95-8985-4b27-8ea5-114b377274ff None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:07:41.668197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 603/1200] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:07:41 2022] GET /identity/v3/role_assignments?user.id=9670f23ecbc14c3a9dc602fcbc32a441&scope.project.id=ad717b2ddd324dbd9f727cf512b99e6b&role.id=5c672a1b2979444383c587a215adeb35 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:07:51.486243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=30 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:07:51.665143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=84 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:39.383235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf469f-43cc-43a3-a3f4-b8e80046c7ea None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:39.383871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf469f-43cc-43a3-a3f4-b8e80046c7ea None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:39.383871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf469f-43cc-43a3-a3f4-b8e80046c7ea None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:39.385594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 598/1201] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:39.391810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403fdd8-bfc9-4d5d-ba9e-f9b4cc0b2435 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:39.392546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403fdd8-bfc9-4d5d-ba9e-f9b4cc0b2435 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:39.392546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403fdd8-bfc9-4d5d-ba9e-f9b4cc0b2435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:39.413690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4403fdd8-bfc9-4d5d-ba9e-f9b4cc0b2435 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:39.413690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4403fdd8-bfc9-4d5d-ba9e-f9b4cc0b2435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:39.514456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 604/1202] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:08:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:08:39.542148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf469f-43cc-43a3-a3f4-b8e80046c7ea None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:39.708288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf469f-43cc-43a3-a3f4-b8e80046c7ea None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:39.758774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:39.760207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:39.760207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:39.760207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:39.777102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:39.778803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:39.811298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:39.812487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:39.813563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:39.813563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:39.813563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:39.813563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:39.817885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 599/1203] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:39.819619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403fdd8-bfc9-4d5d-ba9e-f9b4cc0b2435 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:39.979844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403fdd8-bfc9-4d5d-ba9e-f9b4cc0b2435 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:40.022390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:40.030043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9674137-53fe-406b-b1ef-3b003406efd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:40.030724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9674137-53fe-406b-b1ef-3b003406efd7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:40.031003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9674137-53fe-406b-b1ef-3b003406efd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:40.031278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9674137-53fe-406b-b1ef-3b003406efd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:40.032014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9674137-53fe-406b-b1ef-3b003406efd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:40.034162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9674137-53fe-406b-b1ef-3b003406efd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:40.034162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9674137-53fe-406b-b1ef-3b003406efd7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:40.057997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 605/1204] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:40.059381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:40.161861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:40.229136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69c26f-4b18-4d62-baf2-5594f84a085e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:40.278044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9321a10-c838-435d-9678-95115236a653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:40.278704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9321a10-c838-435d-9678-95115236a653 None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:40.279046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9321a10-c838-435d-9678-95115236a653 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:40.279307 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9321a10-c838-435d-9678-95115236a653 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:40.280031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9321a10-c838-435d-9678-95115236a653 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:40.281681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9321a10-c838-435d-9678-95115236a653 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:40.311792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 600/1205] 10.222.0.71 () {60 vars in 1132 bytes} [Mon Jul 18 14:08:40 2022] POST /identity/v3/users => generated 251 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:08:42.628219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92bc0e-6223-4a7f-9503-72a76afea58b None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.628890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92bc0e-6223-4a7f-9503-72a76afea58b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.628991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b92bc0e-6223-4a7f-9503-72a76afea58b None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.630449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 606/1206] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:08:42.639680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7934-19e4-4745-a570-98b48c6c35c6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.639680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7934-19e4-4745-a570-98b48c6c35c6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.639680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad7934-19e4-4745-a570-98b48c6c35c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.659166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-49ad7934-19e4-4745-a570-98b48c6c35c6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:42.660791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-49ad7934-19e4-4745-a570-98b48c6c35c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:42.738095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 601/1207] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:08:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:08:42.762029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92bc0e-6223-4a7f-9503-72a76afea58b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:42.890940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b92bc0e-6223-4a7f-9503-72a76afea58b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:42.943807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4971134-4579-4c97-81b9-750a388e7ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:42.944932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4971134-4579-4c97-81b9-750a388e7ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:42.945255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4971134-4579-4c97-81b9-750a388e7ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:42.945751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4971134-4579-4c97-81b9-750a388e7ad8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:42.961013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4971134-4579-4c97-81b9-750a388e7ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d4971134-4579-4c97-81b9-750a388e7ad8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:08:42.969587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:42.970963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:42.971834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 607/1208] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:08:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:42.978737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad7934-19e4-4745-a570-98b48c6c35c6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.035076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:43.105917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad7934-19e4-4745-a570-98b48c6c35c6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.146562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.147369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.147677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.147955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.148818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:43.149204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.151295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:43.163885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 602/1209] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:08:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:43.170902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4971134-4579-4c97-81b9-750a388e7ad8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.233779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:43.299632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4971134-4579-4c97-81b9-750a388e7ad8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.342888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.343877 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.344298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.344661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.359885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.361969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:43.383601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:43.384584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:43.386244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:43.386244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:43.386244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:43.386244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:43.386244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 608/1210] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:43.390712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.525457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:43.528301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bf3a6-49f9-4266-851d-b9bc26da66db None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.583338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.584211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.584434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.584799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.585711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:43.585994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.587609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:43.603533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 603/1211] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:43.610012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.727294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99abc7d-c864-4c7e-bcc2-d7b0417746a7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.728889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:43.765819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.766609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.766938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.767189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.775785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.777150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:08:43.794008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:43.795860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:43.796821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 609/1212] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:43.802281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:43.930215 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248d0fb-56e4-4aba-b867-302f9088ad8c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:43.970468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:43.971536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:43.972218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:43.972711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:43.973795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:43.974065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:43.975839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:44.004224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 604/1213] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 18 14:08:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:08:44.010798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:44.147005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81710087-73c6-43a3-8725-057fc22fd2a2 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:44.189049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac551db9-43f6-47fe-b1aa-5690d4a41a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:44.189751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac551db9-43f6-47fe-b1aa-5690d4a41a8c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:44.190044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac551db9-43f6-47fe-b1aa-5690d4a41a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:44.190324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac551db9-43f6-47fe-b1aa-5690d4a41a8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:44.244273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:44.295937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac551db9-43f6-47fe-b1aa-5690d4a41a8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:44.297584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac551db9-43f6-47fe-b1aa-5690d4a41a8c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:44.437946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 610/1214] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:08:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 18 14:08:44.440011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:44.464092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:44.573181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4358a89-a58d-4939-81f4-1dd637abe963 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:44.617477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:44.618521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:44.618869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:44.619263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:44.632478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:44.634111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:44.664391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:44.666075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:44.667122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:44.667122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:44.667122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:44.667122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:44.671043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 605/1215] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:44.672164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac551db9-43f6-47fe-b1aa-5690d4a41a8c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:44.748537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:44.793482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac551db9-43f6-47fe-b1aa-5690d4a41a8c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:44.838042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:44.838876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:44.839251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:44.839627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:44.840479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:44.840933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:44.842575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:44.856541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 611/1216] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:44.862996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:44.939027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:44.972493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4af96-4cd2-4e8a-8372-3f73535cf2c6 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:45.012483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:45.013231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:45.013683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:45.014008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:45.022963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:45.024851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:08:45.042300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:45.043631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:45.044681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:08:45.044681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:08:45.044681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:08:45.044681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:08:45.044681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:08:45.044681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:08:45.044681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:08:45.044681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:45.049482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 606/1217] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:08:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:45.051286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:45.181094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11903325-ca0e-4276-be79-e1216bbd185a None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:45.209522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:45.227028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8de7ff-556d-43c7-b5ed-d7b2a7f272c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:45.227820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8de7ff-556d-43c7-b5ed-d7b2a7f272c1 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:45.228014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8de7ff-556d-43c7-b5ed-d7b2a7f272c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:45.228286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8de7ff-556d-43c7-b5ed-d7b2a7f272c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:45.229170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8de7ff-556d-43c7-b5ed-d7b2a7f272c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:45.229482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8de7ff-556d-43c7-b5ed-d7b2a7f272c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:45.231202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8de7ff-556d-43c7-b5ed-d7b2a7f272c1 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:45.246975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 612/1218] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:08:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:45.255495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:45.393766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b18ab-b9cc-4415-9312-7376d798b126 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:45.426239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:45.433969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821139f-1b65-4ff2-95ed-cc26be4ac8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:45.434643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821139f-1b65-4ff2-95ed-cc26be4ac8ee None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:45.434977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821139f-1b65-4ff2-95ed-cc26be4ac8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:45.435232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821139f-1b65-4ff2-95ed-cc26be4ac8ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:45.436216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821139f-1b65-4ff2-95ed-cc26be4ac8ee None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=8895e6cf7a1e445c825f391309766d76) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:45.436583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821139f-1b65-4ff2-95ed-cc26be4ac8ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:45.438336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821139f-1b65-4ff2-95ed-cc26be4ac8ee None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:45.458095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 607/1219] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:08:45 2022] GET /identity/v3/role_assignments?user.id=8895e6cf7a1e445c825f391309766d76&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:47.614783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ef555-3784-4e97-b592-4fd4b7a856b8 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:47.615340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ef555-3784-4e97-b592-4fd4b7a856b8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:47.615340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ef555-3784-4e97-b592-4fd4b7a856b8 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:47.617225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 613/1220] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:47.623265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f57a-a03f-4164-9868-61a148ade407 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:47.623902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f57a-a03f-4164-9868-61a148ade407 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:47.623902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528f57a-a03f-4164-9868-61a148ade407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:47.643911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8528f57a-a03f-4164-9868-61a148ade407 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:47.645292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8528f57a-a03f-4164-9868-61a148ade407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:47.728680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 608/1221] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:08:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:08:47.754540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ef555-3784-4e97-b592-4fd4b7a856b8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:47.877855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ef555-3784-4e97-b592-4fd4b7a856b8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:47.919869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4f844-88e0-4f0c-83d5-d82db9e0cf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:47.920663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f844-88e0-4f0c-83d5-d82db9e0cf4d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:47.920951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f844-88e0-4f0c-83d5-d82db9e0cf4d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:47.921226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4f844-88e0-4f0c-83d5-d82db9e0cf4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:47.933307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4f844-88e0-4f0c-83d5-d82db9e0cf4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8dc4f844-88e0-4f0c-83d5-d82db9e0cf4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:08:47.941906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:47.942966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:47.943845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 614/1222] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:47.949685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f57a-a03f-4164-9868-61a148ade407 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:48.062505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528f57a-a03f-4164-9868-61a148ade407 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:48.090188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:48.103053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:48.103757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:48.104048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:48.104409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:48.105331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] RBAC: Adding query filter params (name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:48.105756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:48.107470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:48.117386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 609/1223] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:08:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:08:48.123803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4f844-88e0-4f0c-83d5-d82db9e0cf4d None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:48.181934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:48.255816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4f844-88e0-4f0c-83d5-d82db9e0cf4d None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:48.301232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:48.302099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:48.302383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:48.302662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:48.315503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:48.317107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:48.336179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:48.337409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:48.343329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:48.343329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:48.343329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:48.343329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:48.343329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 615/1224] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:08:48.344775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:48.456731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88394df6-e1f6-49a0-b34e-b8afd5c67193 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:48.494369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:48.495043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:48.495328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:48.495603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:48.496327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:48.496635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:48.498047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:48.511466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 610/1225] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:48.517759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:48.630685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:48.637700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e780a0a-3cc4-4abc-a63d-4ce0b4c04c2c None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:48.676134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:48.677000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:48.677285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:48.677565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:48.686310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:48.687847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:08:48.703208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:48.704449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:48.705450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 616/1226] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:48.711351 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:48.765335 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:48.832759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ba800-e496-45db-87a1-cf1d864806e7 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:48.875583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:48.876274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:48.876554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:48.876829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:48.877698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:48.878017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:48.879474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:48.901289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 611/1227] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 18 14:08:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:48.913437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.035442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f76e9-6986-426b-b4af-b8b5d401aa84 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.074553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.075362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.075634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.075901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.087673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.089145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:49.108855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:49.110313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:49.111223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:49.111223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:49.111223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:49.111223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:49.111223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 617/1228] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:49.118055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.226017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2663a63-a8e9-47ed-be17-5d6f92fca71e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.229355 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:49.275301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.276103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.276417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.276771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.277610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:49.277959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.279464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.293816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 612/1229] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:49.301162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.312208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:49.418468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc834180-4b8a-4ffc-8265-cb2914af24cf None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.459066 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.459919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.460313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.460582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.468740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.470428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:08:49.485734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:49.486759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:49.488040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 618/1230] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:49.494219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.612842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f21b58-f9f0-4d4c-8663-447046b068e4 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.654109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adfede-bdd6-4b44-b500-099152e7ccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.654813 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adfede-bdd6-4b44-b500-099152e7ccf9 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.655097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adfede-bdd6-4b44-b500-099152e7ccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.655434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adfede-bdd6-4b44-b500-099152e7ccf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.656283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adfede-bdd6-4b44-b500-099152e7ccf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:49.656611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adfede-bdd6-4b44-b500-099152e7ccf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.658270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adfede-bdd6-4b44-b500-099152e7ccf9 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 613/1231] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:49.679636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:49.771751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:49.810965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535073b-9f96-4713-9c2c-79752fb4ab93 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:49.851328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a652d5b-a05d-4040-81bb-a0cb74aa099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:49.852226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a652d5b-a05d-4040-81bb-a0cb74aa099c None admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:49.852520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a652d5b-a05d-4040-81bb-a0cb74aa099c None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:49.852797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a652d5b-a05d-4040-81bb-a0cb74aa099c None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/8895e6cf7a1e445c825f391309766d76/roles/cc93d9b4856a4b1cbf172a716c54860a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:49.869407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a652d5b-a05d-4040-81bb-a0cb74aa099c None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=8895e6cf7a1e445c825f391309766d76, role_id=cc93d9b4856a4b1cbf172a716c54860a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:49.870885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a652d5b-a05d-4040-81bb-a0cb74aa099c None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:49.885820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 619/1232] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:08:49 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/8895e6cf7a1e445c825f391309766d76/roles/cc93d9b4856a4b1cbf172a716c54860a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:08:51.958062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5730d-ea8b-46fa-b8f9-c84a104d7ef1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:51.958680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5730d-ea8b-46fa-b8f9-c84a104d7ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:51.958680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5730d-ea8b-46fa-b8f9-c84a104d7ef1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:51.959927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 614/1233] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:51.966501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4821c-a24b-4589-a574-4e73dc0c9a2a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:51.967002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4821c-a24b-4589-a574-4e73dc0c9a2a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:51.967104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4821c-a24b-4589-a574-4e73dc0c9a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:51.986192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-64a4821c-a24b-4589-a574-4e73dc0c9a2a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:51.987650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-64a4821c-a24b-4589-a574-4e73dc0c9a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:52.062082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 620/1234] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:08:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:08:52.087819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5730d-ea8b-46fa-b8f9-c84a104d7ef1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:52.117551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:52.200521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5730d-ea8b-46fa-b8f9-c84a104d7ef1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:52.253645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01319503-4129-402d-9c29-19ac8702fc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:52.254448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01319503-4129-402d-9c29-19ac8702fc0b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.254814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01319503-4129-402d-9c29-19ac8702fc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.255142 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01319503-4129-402d-9c29-19ac8702fc0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.266889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01319503-4129-402d-9c29-19ac8702fc0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-01319503-4129-402d-9c29-19ac8702fc0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:08:52.275428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:52.276960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 18 14:08:52.278178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:52.285393 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 615/1235] 10.222.0.71 () {56 vars in 1102 bytes} [Mon Jul 18 14:08:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:52.285564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4821c-a24b-4589-a574-4e73dc0c9a2a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:52.334712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:52.394334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4821c-a24b-4589-a574-4e73dc0c9a2a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:52.434569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:52.435270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.435550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.435825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.436567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] RBAC: Adding query filter params (name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:52.436873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:52.438293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:52.451629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 621/1236] 10.222.0.71 () {56 vars in 1103 bytes} [Mon Jul 18 14:08:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:52.452939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01319503-4129-402d-9c29-19ac8702fc0b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:52.549722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:52.569697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01319503-4129-402d-9c29-19ac8702fc0b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:52.607733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:52.608518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.608802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.609080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.621213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:52.622725 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:52.640394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:52.641500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:52.643060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:52.643060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:52.643060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:52.643060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:52.643060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 616/1237] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:52.648147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:52.759753 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e6b0c-d8ae-43fe-b232-251ce01b5365 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:52.769228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:52.795812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:52.796632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.797014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.797381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:52.798260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:52.798636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:52.800229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:52.814567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 622/1238] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:52.820860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:52.943782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1970e4a-0fd0-47f5-9ed5-dba4b0671e26 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:52.997478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:52.998655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:52.999124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:52.999586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:53.004095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:53.014021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:53.017236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:08:53.038290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:53.039567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:53.045776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 617/1239] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:53.046508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:53.181897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8571cc1d-bd0a-4bb4-8c53-0f97b4153273 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:53.222592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:53.223281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:53.223567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:53.223846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:53.224787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:53.225121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:53.226613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:53.249031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 623/1240] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 18 14:08:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:53.255704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:53.376925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cf4e5-5142-45ec-8958-628656471f18 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:53.419657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390f4ca-be9e-4651-8f7e-917e1c1ac336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:53.420336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390f4ca-be9e-4651-8f7e-917e1c1ac336 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:53.420619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390f4ca-be9e-4651-8f7e-917e1c1ac336 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:53.420892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390f4ca-be9e-4651-8f7e-917e1c1ac336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:53.508841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:53.517372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390f4ca-be9e-4651-8f7e-917e1c1ac336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:53.520234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390f4ca-be9e-4651-8f7e-917e1c1ac336 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:53.647525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 618/1241] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:08:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:08:53.654683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:53.699296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:53.798142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e164f-c532-4620-b976-ae244f7d0caa None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:53.846913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:53.847958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:53.848284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:53.848562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:53.860544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:53.862134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:53.887590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:53.888898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:53.890029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:53.890029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:53.890029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:53.890029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:53.890029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 624/1242] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:53.896024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390f4ca-be9e-4651-8f7e-917e1c1ac336 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:54.001614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:54.008406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390f4ca-be9e-4651-8f7e-917e1c1ac336 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:54.051422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:54.052090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:54.052444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:54.052918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:54.053834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:54.054154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:54.055762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:54.069058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 619/1243] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:08:54.075497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:54.168266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:54.194281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70cc73-bf42-4d15-862c-5a96cf8f1a66 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:54.240533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:54.241993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:54.242399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:54.242779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:54.251875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:54.253631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:08:54.274000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:54.275387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:54.276633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:08:54.276633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:08:54.276633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:08:54.276633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:08:54.276633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:08:54.276633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:08:54.276633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:08:54.276633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:54.277999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 625/1244] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:08:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:54.284789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:54.420773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d9864-bc88-4d8f-9bb8-ccc4abb9fda3 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:54.464238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f46ae-ec53-4aeb-ac79-b4e4310ce321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:54.465097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f46ae-ec53-4aeb-ac79-b4e4310ce321 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:54.465379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f46ae-ec53-4aeb-ac79-b4e4310ce321 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:54.465621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f46ae-ec53-4aeb-ac79-b4e4310ce321 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:54.466612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f46ae-ec53-4aeb-ac79-b4e4310ce321 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:54.466953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f46ae-ec53-4aeb-ac79-b4e4310ce321 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:54.468721 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f46ae-ec53-4aeb-ac79-b4e4310ce321 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:54.485238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 620/1245] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:08:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:54.489806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:54.608883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7a3ac-af5d-40e7-9ee5-1bfc2d75864e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:54.652746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed66bc-9139-4621-a44f-3d2a68f91660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:54.653449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed66bc-9139-4621-a44f-3d2a68f91660 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:54.653784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed66bc-9139-4621-a44f-3d2a68f91660 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:54.654059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed66bc-9139-4621-a44f-3d2a68f91660 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:54.654854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed66bc-9139-4621-a44f-3d2a68f91660 None admin] RBAC: Adding query filter params (role.id=cc93d9b4856a4b1cbf172a716c54860a, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=8895e6cf7a1e445c825f391309766d76) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:54.655176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed66bc-9139-4621-a44f-3d2a68f91660 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:54.656643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed66bc-9139-4621-a44f-3d2a68f91660 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:54.666133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 626/1246] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:08:54 2022] GET /identity/v3/role_assignments?user.id=8895e6cf7a1e445c825f391309766d76&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=cc93d9b4856a4b1cbf172a716c54860a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:08:56.887089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485d33c-6b81-4148-a362-f7cdfafc550a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.887719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485d33c-6b81-4148-a362-f7cdfafc550a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.887719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485d33c-6b81-4148-a362-f7cdfafc550a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.889327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 621/1247] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:08:56.895394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23274f1-febc-4e0b-bc81-85e51f6957f4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:56.895899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23274f1-febc-4e0b-bc81-85e51f6957f4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:56.895899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23274f1-febc-4e0b-bc81-85e51f6957f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:56.903615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:56.917530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c23274f1-febc-4e0b-bc81-85e51f6957f4 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:08:56.918803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c23274f1-febc-4e0b-bc81-85e51f6957f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:08:57.001847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 627/1248] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:08:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:08:57.027957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485d33c-6b81-4148-a362-f7cdfafc550a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.049069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:57.157396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485d33c-6b81-4148-a362-f7cdfafc550a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.211310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21c17-8f7f-4fcd-95f4-ab6a9ec2962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.212191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21c17-8f7f-4fcd-95f4-ab6a9ec2962d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.212560 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21c17-8f7f-4fcd-95f4-ab6a9ec2962d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.212836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21c17-8f7f-4fcd-95f4-ab6a9ec2962d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.230720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a21c17-8f7f-4fcd-95f4-ab6a9ec2962d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-97a21c17-8f7f-4fcd-95f4-ab6a9ec2962d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:08:57.245340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:57.246670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:08:57.247784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:57.249858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 622/1249] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:57.255758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23274f1-febc-4e0b-bc81-85e51f6957f4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.391980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23274f1-febc-4e0b-bc81-85e51f6957f4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.428178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:57.435151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.435860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.436142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.436420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.437173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:57.437484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.438922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.448320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 628/1250] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:57.456812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21c17-8f7f-4fcd-95f4-ab6a9ec2962d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.550316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:57.577878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21c17-8f7f-4fcd-95f4-ab6a9ec2962d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.619525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.620336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.620648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.620885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.635951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.637766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:57.656468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:57.657684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:57.658612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:57.658612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:57.658612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:57.658612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:57.658612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 623/1251] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:57.665065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:57.808174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb34047-05dd-4039-8064-ecc8814788a6 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:57.858220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:57.861862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:57.861862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:57.861862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:57.861862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:57.861862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:57.861862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:57.880909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 629/1252] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:57.888226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.035196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d8e35-3972-40fa-9fd7-829dc50a5e35 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.065165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:58.074023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.074842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.075125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.075402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.085415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.086923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:08:58.103582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:58.104691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:08:58.105641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 624/1253] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:58.111049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.176397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:58.236002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c0109-e583-4ca6-a051-e45070045f40 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.289835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.290966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.291438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.291881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.293051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:58.293585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.295971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.321579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 630/1254] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 18 14:08:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:08:58.328150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.458230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12dd43-b1b5-40c1-bc3e-09a01f16cbaf None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.497883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9adcf1-caf4-4013-88f3-47af62eec1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.498651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9adcf1-caf4-4013-88f3-47af62eec1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.498869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9adcf1-caf4-4013-88f3-47af62eec1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.499156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9adcf1-caf4-4013-88f3-47af62eec1d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.563268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:58.577716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9adcf1-caf4-4013-88f3-47af62eec1d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.579197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9adcf1-caf4-4013-88f3-47af62eec1d2 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.692244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 625/1255] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:08:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:08:58.699248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:58.823163 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e920c-cd90-43e9-b01e-6cdfabc52f12 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:58.841194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:58.870451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:58.871480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:58.871851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:58.872285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:58.885109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:58.886756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:08:58.909130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:58.910429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:58.911567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:08:58.911567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:08:58.911567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:08:58.911567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:58.911567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 631/1256] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:08:58.917441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9adcf1-caf4-4013-88f3-47af62eec1d2 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:59.015606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:59.032271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9adcf1-caf4-4013-88f3-47af62eec1d2 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.079503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:59.080794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.081495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.082430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.083466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:59.083931 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.086925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.109090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 626/1257] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:59.115716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:59.235940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955adaba-1f7c-4dca-8305-c67213255ab8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.280226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:59.280985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.281347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.281690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.290659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.292509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:08:59.312167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:08:59.313380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:08:59.314964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 632/1258] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:08:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:08:59.321415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:59.473221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca145d0-fbb8-411f-a17e-4a86e6cfb96e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.532796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dadb3-436f-43ce-97c3-4ba92ae13229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:59.534012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dadb3-436f-43ce-97c3-4ba92ae13229 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.534528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dadb3-436f-43ce-97c3-4ba92ae13229 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.534917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dadb3-436f-43ce-97c3-4ba92ae13229 None admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.536311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41dadb3-436f-43ce-97c3-4ba92ae13229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:59.536741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41dadb3-436f-43ce-97c3-4ba92ae13229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.539251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41dadb3-436f-43ce-97c3-4ba92ae13229 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.563370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 627/1259] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:08:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:08:59.565063 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:08:59.593852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:08:59.698271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97ef5e-2e91-4335-8db4-36c4a7cc1ec8 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:08:59.738036 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12140274-1b93-4f43-9013-b320317b2f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:08:59.738564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12140274-1b93-4f43-9013-b320317b2f89 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:08:59.738843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12140274-1b93-4f43-9013-b320317b2f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:08:59.739110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12140274-1b93-4f43-9013-b320317b2f89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:08:59.739942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12140274-1b93-4f43-9013-b320317b2f89 None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=8895e6cf7a1e445c825f391309766d76) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:08:59.740246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12140274-1b93-4f43-9013-b320317b2f89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:08:59.742220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12140274-1b93-4f43-9013-b320317b2f89 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:08:59.748760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 633/1260] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:08:59 2022] GET /identity/v3/role_assignments?user.id=8895e6cf7a1e445c825f391309766d76&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=5c672a1b2979444383c587a215adeb35 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:01.810857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d523-eca0-47df-ad67-5e5630e6f72b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.811442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d523-eca0-47df-ad67-5e5630e6f72b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.811442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80d523-eca0-47df-ad67-5e5630e6f72b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.812775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 628/1261] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:01.818488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e291-2939-4b00-9d1c-a1c8af91167b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:01.819072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e291-2939-4b00-9d1c-a1c8af91167b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:01.819072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e291-2939-4b00-9d1c-a1c8af91167b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:01.837011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9c97e291-2939-4b00-9d1c-a1c8af91167b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:01.838258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9c97e291-2939-4b00-9d1c-a1c8af91167b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:01.915546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 634/1262] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:09:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:09:01.938366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80d523-eca0-47df-ad67-5e5630e6f72b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:01.952682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:02.051000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80d523-eca0-47df-ad67-5e5630e6f72b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.087250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1dcb3-a335-476e-b8b7-d4cc4ec45d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.088068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1dcb3-a335-476e-b8b7-d4cc4ec45d9d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.088350 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1dcb3-a335-476e-b8b7-d4cc4ec45d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.088608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1dcb3-a335-476e-b8b7-d4cc4ec45d9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.100157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1dcb3-a335-476e-b8b7-d4cc4ec45d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-65f1dcb3-a335-476e-b8b7-d4cc4ec45d9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:09:02.108599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:02.109874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:09:02.110781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 629/1263] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:09:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:02.118453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97e291-2939-4b00-9d1c-a1c8af91167b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.139866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:02.231949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97e291-2939-4b00-9d1c-a1c8af91167b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.276334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.277024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.277312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.277592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.278500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:02.278803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.280266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.289686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 635/1264] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:09:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:09:02.298299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1dcb3-a335-476e-b8b7-d4cc4ec45d9d None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.371464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:02.417104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1dcb3-a335-476e-b8b7-d4cc4ec45d9d None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.466952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.467739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.468042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.468305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.479624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.481215 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:09:02.499337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:02.500419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:02.501497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:09:02.501497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:09:02.501497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:09:02.501497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:09:02.501497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 630/1265] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:02.507729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.599709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:02.622125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ec2a6-69fd-4c63-9e1b-44794803ef97 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.667149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.667819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.668107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.668385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.669112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:02.669422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.670899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.684580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 636/1266] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:02.690796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.801595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efa902-f13f-4105-ad1e-deb36df475b0 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:02.822339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:02.840433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:02.841417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:02.841754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:02.842030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:02.851484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:02.852972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:09:02.869074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:02.870320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:09:02.871569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 631/1267] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:09:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:02.876596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:02.988134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9888e-908f-4aac-94ff-d509f6470c73 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:03.025384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:03.026090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:03.026370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:03.026644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:03.027535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:03.027826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:03.029267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:03.039704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:03.050252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 637/1268] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 18 14:09:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:03.056464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:03.172241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a15b1d-4991-4ed5-8baf-5c35d1ced8b8 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:03.210014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:03.210803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:03.211089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:03.211482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:03.223792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:03.225265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:09:03.244216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:03.245226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:03.246187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:09:03.246187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:09:03.246187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:09:03.246187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:09:03.246187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 632/1269] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:03.254417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:03.373014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e57c6-a03a-4aca-81e0-928a0bbe4207 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:03.411357 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:03.412031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:03.412313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:03.412589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:03.413320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:03.413628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:03.415104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:03.428019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 638/1270] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:09:03.434425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:03.467951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:03.539711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119719d-cd71-46d6-87b0-fe2e61d48d1e None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:03.575632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:03.576432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:03.576711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:03.576987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:03.584351 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:03.585906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:09:03.600574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:03.601627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:09:03.602723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 633/1271] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:09:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:03.609706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:03.670825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:03.718432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdd8b03-2467-4966-94aa-ce71237c05d5 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:03.754885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c696369-a7db-4413-8eb2-a0f855d9e85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:03.755522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c696369-a7db-4413-8eb2-a0f855d9e85f None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:03.755800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c696369-a7db-4413-8eb2-a0f855d9e85f None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:03.756065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c696369-a7db-4413-8eb2-a0f855d9e85f None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:03.756846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c696369-a7db-4413-8eb2-a0f855d9e85f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:03.757151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c696369-a7db-4413-8eb2-a0f855d9e85f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:03.758620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c696369-a7db-4413-8eb2-a0f855d9e85f None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:03.772106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 639/1272] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:09:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:03.778405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:03.878703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:03.888933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8891ec-a975-40dc-84f2-050d94073e3b None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:03.925167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaf327-06da-4fd6-842f-df5cf757602f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:03.926096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf327-06da-4fd6-842f-df5cf757602f None admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:03.926376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf327-06da-4fd6-842f-df5cf757602f None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:03.926662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaf327-06da-4fd6-842f-df5cf757602f None admin] PATH_INFO: `/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/8895e6cf7a1e445c825f391309766d76/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:03.928806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaf327-06da-4fd6-842f-df5cf757602f None admin] RBAC: Authorizing `identity:create_grant(project_id=83aa8da005274151aa6d73a3e6fbaa7c, user_id=8895e6cf7a1e445c825f391309766d76, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:03.930313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aaf327-06da-4fd6-842f-df5cf757602f None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:03.944891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 634/1273] 10.222.0.71 () {58 vars in 1439 bytes} [Mon Jul 18 14:09:03 2022] PUT /identity/v3/projects/83aa8da005274151aa6d73a3e6fbaa7c/users/8895e6cf7a1e445c825f391309766d76/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:09:05.944202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b11ff-026d-4510-9156-5e6e2158beb3 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.944732 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b11ff-026d-4510-9156-5e6e2158beb3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.944732 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b11ff-026d-4510-9156-5e6e2158beb3 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.946229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 640/1274] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:05.952012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbaaf-1468-4633-a047-e503fd320d3c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:05.952012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbaaf-1468-4633-a047-e503fd320d3c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:05.953151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebbaaf-1468-4633-a047-e503fd320d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:05.971039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-97ebbaaf-1468-4633-a047-e503fd320d3c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:05.972470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-97ebbaaf-1468-4633-a047-e503fd320d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:06.047994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 635/1275] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:09:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:09:06.074040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b11ff-026d-4510-9156-5e6e2158beb3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.194791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b11ff-026d-4510-9156-5e6e2158beb3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.214418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:06.239422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c787e2d-d9aa-4612-bcab-2bf5027d7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.240421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c787e2d-d9aa-4612-bcab-2bf5027d7a79 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.240710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c787e2d-d9aa-4612-bcab-2bf5027d7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.241364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c787e2d-d9aa-4612-bcab-2bf5027d7a79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.256962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c787e2d-d9aa-4612-bcab-2bf5027d7a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-5c787e2d-d9aa-4612-bcab-2bf5027d7a79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 18 14:09:06.265468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:06.266720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:06.268139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 641/1276] 10.222.0.71 () {56 vars in 1096 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:06.274413 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbaaf-1468-4633-a047-e503fd320d3c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.378845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:06.401204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebbaaf-1468-4633-a047-e503fd320d3c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.449569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.450212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.450498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.450776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.451548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:06.452034 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.453463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.463554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 636/1277] 10.222.0.71 () {56 vars in 1099 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:06.469774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c787e2d-d9aa-4612-bcab-2bf5027d7a79 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.578688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c787e2d-d9aa-4612-bcab-2bf5027d7a79 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.617910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.618713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.618993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.619266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.631241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.632775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:09:06.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:06.653243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:06.654430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:09:06.654430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:09:06.654430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:09:06.654430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:06.654430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 642/1278] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:06.659450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.773360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e91f0d-ce51-4b1f-9f28-6e124b0e81de None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.814407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:06.815176 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:06.815419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:06.815698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:06.816434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:06.816747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:06.818180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:06.832622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 637/1279] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:06.838892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:06.877254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:06.961576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722256f-5916-45b6-979d-bf563c784cb7 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:06.999860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.000664 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.000950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.001211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.010107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.011567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 18 14:09:07.027315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:07.028482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 18 14:09:07.029772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:07.033462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 643/1280] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:09:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 18 14:09:07.035117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.039193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:07.150601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17c5c3-c169-4436-9918-1d5f5d92fc24 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.186996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.187671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.187951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.188221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.189058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:07.189369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.191133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.210772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 638/1281] 10.222.0.71 () {56 vars in 1143 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:07.217077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.328058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:07.338918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb136a9-577f-4821-8f39-fcf4803c1739 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.394220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79455d2d-c4c4-443d-9bba-72c50a88a18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.395340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79455d2d-c4c4-443d-9bba-72c50a88a18e None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.395847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79455d2d-c4c4-443d-9bba-72c50a88a18e None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.396309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79455d2d-c4c4-443d-9bba-72c50a88a18e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.478899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79455d2d-c4c4-443d-9bba-72c50a88a18e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.480409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79455d2d-c4c4-443d-9bba-72c50a88a18e None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.578709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:07.591279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 644/1282] 10.222.0.71 () {58 vars in 1282 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 18 14:09:07.597948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.673179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:07.713088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657f14bb-a8f7-433c-8946-50348d4ea6ad None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.750189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.751007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.751324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.751604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.763219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.764690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 18 14:09:07.783895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:07.785454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:07.786575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 18 14:09:07.786575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 18 14:09:07.786575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 18 14:09:07.786575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:09:07.786575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 639/1283] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:07.791561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79455d2d-c4c4-443d-9bba-72c50a88a18e None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:07.899958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79455d2d-c4c4-443d-9bba-72c50a88a18e None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:07.936579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:07.937212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:07.937521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:07.937825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:07.938547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:07.938854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:07.940585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:07.952269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 645/1284] 10.222.0.71 () {56 vars in 1109 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:07.959843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.118619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e2bf5-c7b2-4bff-bd1b-7399f03cce57 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.143657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:08.166272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.167002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.167290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.167572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.175565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.177042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 18 14:09:08.192851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:08.194066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: ERROR keystone.server.flask.application Jul 18 14:09:08.195126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 640/1285] 10.222.0.71 () {56 vars in 1111 bytes} [Mon Jul 18 14:09:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:08.200721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.260853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:08.320613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5338b33-d6aa-4b04-9d6c-b71cdec10017 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.357318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acbf2a-c14f-462d-9d0e-ac352445a984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.358016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acbf2a-c14f-462d-9d0e-ac352445a984 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.358300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acbf2a-c14f-462d-9d0e-ac352445a984 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.358571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acbf2a-c14f-462d-9d0e-ac352445a984 None admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.359376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acbf2a-c14f-462d-9d0e-ac352445a984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:08.359691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acbf2a-c14f-462d-9d0e-ac352445a984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.361306 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acbf2a-c14f-462d-9d0e-ac352445a984 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.376839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 646/1286] 10.222.0.71 () {56 vars in 1148 bytes} [Mon Jul 18 14:09:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:08.383559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:08.496706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b4eda-042d-4735-83a3-f9f3a67eef79 None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:08.532410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9df0dd-d28b-4fca-8d87-6bad0a548b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:08.533090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9df0dd-d28b-4fca-8d87-6bad0a548b5d None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:08.533375 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9df0dd-d28b-4fca-8d87-6bad0a548b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:08.533756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9df0dd-d28b-4fca-8d87-6bad0a548b5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:08.534677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9df0dd-d28b-4fca-8d87-6bad0a548b5d None admin] RBAC: Adding query filter params (role.id=5c672a1b2979444383c587a215adeb35, scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c, user.id=8895e6cf7a1e445c825f391309766d76) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:08.534918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9df0dd-d28b-4fca-8d87-6bad0a548b5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:08.536375 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9df0dd-d28b-4fca-8d87-6bad0a548b5d None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:08.542887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 641/1287] 10.222.0.71 () {56 vars in 1374 bytes} [Mon Jul 18 14:09:08 2022] GET /identity/v3/role_assignments?user.id=8895e6cf7a1e445c825f391309766d76&scope.project.id=83aa8da005274151aa6d73a3e6fbaa7c&role.id=5c672a1b2979444383c587a215adeb35 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:10.640103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06161d-f107-439f-a483-13de14c3a818 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.640729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06161d-f107-439f-a483-13de14c3a818 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.640729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06161d-f107-439f-a483-13de14c3a818 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.642033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 647/1288] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:10.648629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158566f-562e-431d-9d25-80fde3f5015c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.649261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158566f-562e-431d-9d25-80fde3f5015c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.649261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5158566f-562e-431d-9d25-80fde3f5015c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.670016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5158566f-562e-431d-9d25-80fde3f5015c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:10.671267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5158566f-562e-431d-9d25-80fde3f5015c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:10.711755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:10.756229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 642/1289] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:09:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:09:10.779426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06161d-f107-439f-a483-13de14c3a818 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:10.898397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06161d-f107-439f-a483-13de14c3a818 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:10.910249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:10.936587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:10.937560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:10.937871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:10.938137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:10.938798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:10.940905 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:10.953344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:10.954639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:10.956011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 648/1290] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:09:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:10.961050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158566f-562e-431d-9d25-80fde3f5015c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.073929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5158566f-562e-431d-9d25-80fde3f5015c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.109619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f9176-a2e9-4ad2-8f86-e6c1ed07546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.110303 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f9176-a2e9-4ad2-8f86-e6c1ed07546e None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.110593 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f9176-a2e9-4ad2-8f86-e6c1ed07546e None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.110867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f9176-a2e9-4ad2-8f86-e6c1ed07546e None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.111587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f9176-a2e9-4ad2-8f86-e6c1ed07546e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:11.112010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f9176-a2e9-4ad2-8f86-e6c1ed07546e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.113457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f9176-a2e9-4ad2-8f86-e6c1ed07546e None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.118739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 643/1291] 10.222.0.71 () {56 vars in 1116 bytes} [Mon Jul 18 14:09:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:11.124828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:11.235606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c389d-1e66-4836-807c-43112313cec4 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:11.282473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f464932d-f5b5-4e0e-ab43-fd9dad8273de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:11.283332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464932d-f5b5-4e0e-ab43-fd9dad8273de None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:11.283620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464932d-f5b5-4e0e-ab43-fd9dad8273de None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:11.284164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f464932d-f5b5-4e0e-ab43-fd9dad8273de None admin] PATH_INFO: `/v3/services` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:11.284691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464932d-f5b5-4e0e-ab43-fd9dad8273de None admin] RBAC: Adding query filter params (type=placement) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:11.285041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464932d-f5b5-4e0e-ab43-fd9dad8273de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:11.286540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f464932d-f5b5-4e0e-ab43-fd9dad8273de None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:11.292605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 649/1292] 10.222.0.71 () {56 vars in 1116 bytes} [Mon Jul 18 14:09:11 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:13.432940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa5c8b-858f-4c75-ad0c-24593acdf6b5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:13.433534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa5c8b-858f-4c75-ad0c-24593acdf6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:13.433534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa5c8b-858f-4c75-ad0c-24593acdf6b5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:13.434968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 644/1293] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:13.440427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3e74-7a71-4634-a7d6-30ca504f69b1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:13.440869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3e74-7a71-4634-a7d6-30ca504f69b1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:13.440962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f3e74-7a71-4634-a7d6-30ca504f69b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:13.460768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b53f3e74-7a71-4634-a7d6-30ca504f69b1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:13.462014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b53f3e74-7a71-4634-a7d6-30ca504f69b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:13.496961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:13.538361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 650/1294] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:09:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:09:13.565458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa5c8b-858f-4c75-ad0c-24593acdf6b5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:13.604925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:13.687916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa5c8b-858f-4c75-ad0c-24593acdf6b5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:13.725180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6432a65-5715-4ae5-b0c7-258d161cb64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:13.725877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6432a65-5715-4ae5-b0c7-258d161cb64b None admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:13.726159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6432a65-5715-4ae5-b0c7-258d161cb64b None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:13.726433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6432a65-5715-4ae5-b0c7-258d161cb64b None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:13.727151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6432a65-5715-4ae5-b0c7-258d161cb64b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:13.728516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6432a65-5715-4ae5-b0c7-258d161cb64b None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:13.745314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 645/1295] 10.222.0.71 () {60 vars in 1141 bytes} [Mon Jul 18 14:09:13 2022] POST /identity/v3/services => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:15.794117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48ee3-e92b-45d3-92ca-4528b9039365 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.794678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48ee3-e92b-45d3-92ca-4528b9039365 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.794678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f48ee3-e92b-45d3-92ca-4528b9039365 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.796062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 651/1296] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:15.802801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9b9a-8c8a-4fce-b231-2a1553e42e97 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:15.803584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9b9a-8c8a-4fce-b231-2a1553e42e97 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:15.803736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f9b9a-8c8a-4fce-b231-2a1553e42e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:15.832386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ab6f9b9a-8c8a-4fce-b231-2a1553e42e97 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:15.833623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ab6f9b9a-8c8a-4fce-b231-2a1553e42e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:15.915189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 646/1297] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:09:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:09:15.941882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48ee3-e92b-45d3-92ca-4528b9039365 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.072016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f48ee3-e92b-45d3-92ca-4528b9039365 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.110446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.111248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.111616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.111821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.112654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.114085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:16.126415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:16.127607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:16.128693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 652/1298] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:09:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:16.134237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f9b9a-8c8a-4fce-b231-2a1553e42e97 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.159362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:16.250764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f9b9a-8c8a-4fce-b231-2a1553e42e97 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.290101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.290792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.291068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.291419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.292056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:16.292358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.293761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.299599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 647/1299] 10.222.0.71 () {56 vars in 1116 bytes} [Mon Jul 18 14:09:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:09:16.306138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.342880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:16.419822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd453f9-9843-4a94-abba-0c525c6fc654 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.456716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fb024-70dc-4c76-a3a1-9293d4c6196d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.457381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fb024-70dc-4c76-a3a1-9293d4c6196d None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.457698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fb024-70dc-4c76-a3a1-9293d4c6196d None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.457971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fb024-70dc-4c76-a3a1-9293d4c6196d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.458757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fb024-70dc-4c76-a3a1-9293d4c6196d None admin] RBAC: Adding query filter params (interface=public, service_id=d32ff93749e14d2d9dfc7514892df7ba, region_id=RegionOne) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:16.459070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fb024-70dc-4c76-a3a1-9293d4c6196d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.460473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fb024-70dc-4c76-a3a1-9293d4c6196d None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.466592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 653/1300] 10.222.0.71 () {56 vars in 1251 bytes} [Mon Jul 18 14:09:16 2022] GET /identity/v3/endpoints?service_id=d32ff93749e14d2d9dfc7514892df7ba&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:16.472551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:16.561345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:16.581483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29a0238-766a-40ff-b775-fb5dc221563c None admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:16.618493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290544bf-c9d2-4298-818b-97a71f66e4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:16.619171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290544bf-c9d2-4298-818b-97a71f66e4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:16.619458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290544bf-c9d2-4298-818b-97a71f66e4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:16.619790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290544bf-c9d2-4298-818b-97a71f66e4d4 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:16.620431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290544bf-c9d2-4298-818b-97a71f66e4d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:16.621847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290544bf-c9d2-4298-818b-97a71f66e4d4 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:16.629353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 648/1301] 10.222.0.71 () {56 vars in 1087 bytes} [Mon Jul 18 14:09:16 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:09:18.668339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa3330-f370-4b32-a0ea-5d37d908f87d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.668911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa3330-f370-4b32-a0ea-5d37d908f87d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.668911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa3330-f370-4b32-a0ea-5d37d908f87d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.670335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 654/1302] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:09:18.676406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef6b78-d659-4c29-81bf-c672b899cdcc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.676895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef6b78-d659-4c29-81bf-c672b899cdcc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.677017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef6b78-d659-4c29-81bf-c672b899cdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.694879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-68ef6b78-d659-4c29-81bf-c672b899cdcc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:09:18.696072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-68ef6b78-d659-4c29-81bf-c672b899cdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:09:18.771851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 649/1303] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:09:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 18 14:09:18.796550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa3330-f370-4b32-a0ea-5d37d908f87d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:18.899334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:18.906921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa3330-f370-4b32-a0ea-5d37d908f87d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:18.942149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:18.942963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:18.943242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:18.943516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:18.944174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:18.945589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 18 14:09:18.957591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 18 14:09:18.958805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.get_or_create( Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application with Lock( Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self._enter() Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application return self.creator() Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application created_value = creator( Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: ERROR keystone.server.flask.application Jul 18 14:09:18.959644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 655/1304] 10.222.0.71 () {56 vars in 1117 bytes} [Mon Jul 18 14:09:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 18 14:09:18.967417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef6b78-d659-4c29-81bf-c672b899cdcc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:19.058240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:19.083002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef6b78-d659-4c29-81bf-c672b899cdcc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:19.120187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d78724-2dc7-456e-91ed-3c20c5bdadb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:19.120838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d78724-2dc7-456e-91ed-3c20c5bdadb7 None admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.121124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d78724-2dc7-456e-91ed-3c20c5bdadb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.121398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d78724-2dc7-456e-91ed-3c20c5bdadb7 None admin] PATH_INFO: `/v3/services` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.122139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d78724-2dc7-456e-91ed-3c20c5bdadb7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:09:19.122443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d78724-2dc7-456e-91ed-3c20c5bdadb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:19.123821 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d78724-2dc7-456e-91ed-3c20c5bdadb7 None admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:19.129636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 650/1305] 10.222.0.71 () {56 vars in 1116 bytes} [Mon Jul 18 14:09:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:09:19.135669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:09:19.240548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba5e32-d381-4d87-aa3f-4e4a585f4df3 None admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:09:19.276175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddd041-a43b-4301-b2ce-22e62315c5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:09:19.276841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd041-a43b-4301-b2ce-22e62315c5cb None admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:09:19.277127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd041-a43b-4301-b2ce-22e62315c5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:09:19.277406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddd041-a43b-4301-b2ce-22e62315c5cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:09:19.278183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddd041-a43b-4301-b2ce-22e62315c5cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:09:19.279574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddd041-a43b-4301-b2ce-22e62315c5cb None admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:09:19.300053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 656/1306] 10.222.0.71 () {60 vars in 1144 bytes} [Mon Jul 18 14:09:19 2022] POST /identity/v3/endpoints => generated 331 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:09:29.126082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=30 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:09:29.295413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:26.117325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a17f5f-d98c-42ab-a80a-9d01a9346686 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:26.117943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a17f5f-d98c-42ab-a80a-9d01a9346686 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:26.117943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a17f5f-d98c-42ab-a80a-9d01a9346686 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:26.123405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 651/1307] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:26.125043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9fc12-d7f1-4a8c-843c-f9ad76961d4e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:26.125579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9fc12-d7f1-4a8c-843c-f9ad76961d4e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:26.125579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe9fc12-d7f1-4a8c-843c-f9ad76961d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:26.273120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bbe9fc12-d7f1-4a8c-843c-f9ad76961d4e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:26.274379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bbe9fc12-d7f1-4a8c-843c-f9ad76961d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:26.493519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 657/1308] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:10:26.573293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6338f2-0bdd-4cf2-8f23-b44dd62290ba None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:26.573604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6338f2-0bdd-4cf2-8f23-b44dd62290ba None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:26.573899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6338f2-0bdd-4cf2-8f23-b44dd62290ba None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:26.575600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 652/1309] 10.222.0.71 () {54 vars in 965 bytes} [Mon Jul 18 14:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:26.582087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7beeb-59e4-494d-ac8d-643c52ccf23b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:26.582379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7beeb-59e4-494d-ac8d-643c52ccf23b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:26.582654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7beeb-59e4-494d-ac8d-643c52ccf23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:26.709558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-56d7beeb-59e4-494d-ac8d-643c52ccf23b None None] MFA Rules not processed for user `1f1fe084c3164e50bf5d904350925f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:26.710814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-56d7beeb-59e4-494d-ac8d-643c52ccf23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:26.735825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:26.802780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 658/1310] 10.222.0.71 () {58 vars in 1062 bytes} [Mon Jul 18 14:10:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:10:26.811826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6338f2-0bdd-4cf2-8f23-b44dd62290ba None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:26.813936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0f6338f2-0bdd-4cf2-8f23-b44dd62290ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:26.964712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6338f2-0bdd-4cf2-8f23-b44dd62290ba None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:27.073891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fd54d-84f1-4a05-a0fd-1c2f712ebe2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:27.074559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fd54d-84f1-4a05-a0fd-1c2f712ebe2e service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:27.074844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fd54d-84f1-4a05-a0fd-1c2f712ebe2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:27.075267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fd54d-84f1-4a05-a0fd-1c2f712ebe2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:27.077166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bc7fd54d-84f1-4a05-a0fd-1c2f712ebe2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:27.148671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:27.213987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fd54d-84f1-4a05-a0fd-1c2f712ebe2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:27.215613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fd54d-84f1-4a05-a0fd-1c2f712ebe2e service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:27.240215 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 653/1311] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:29.594236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aac76-5d1e-47db-9d99-dcb14025be63 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:29.594828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aac76-5d1e-47db-9d99-dcb14025be63 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:29.594828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aac76-5d1e-47db-9d99-dcb14025be63 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:29.596637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 659/1312] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:29.602340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d8a0b-8844-40c1-95c3-62ff65b8790c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:29.602812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d8a0b-8844-40c1-95c3-62ff65b8790c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:29.602938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d8a0b-8844-40c1-95c3-62ff65b8790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:29.698135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-386d8a0b-8844-40c1-95c3-62ff65b8790c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:29.699263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-386d8a0b-8844-40c1-95c3-62ff65b8790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:29.807346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 654/1313] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:29.850875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6aac76-5d1e-47db-9d99-dcb14025be63 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:29.852450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5f6aac76-5d1e-47db-9d99-dcb14025be63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:29.973153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6aac76-5d1e-47db-9d99-dcb14025be63 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:30.051444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:30.054152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c335f6f-429e-4a46-9970-1bee9f02116a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:30.054819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c335f6f-429e-4a46-9970-1bee9f02116a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:30.055110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c335f6f-429e-4a46-9970-1bee9f02116a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:30.055395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c335f6f-429e-4a46-9970-1bee9f02116a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:30.057751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0c335f6f-429e-4a46-9970-1bee9f02116a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:30.183603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c335f6f-429e-4a46-9970-1bee9f02116a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:30.185058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c335f6f-429e-4a46-9970-1bee9f02116a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:30.205834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 660/1314] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:10:32.629907 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783b73e-78c4-464d-b7f4-f88b25fa83e3 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:32.630697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783b73e-78c4-464d-b7f4-f88b25fa83e3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:32.630697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783b73e-78c4-464d-b7f4-f88b25fa83e3 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:32.632014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 655/1315] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:32.637888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a670d79-da94-415a-bb66-78d13b07bb7c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:32.638348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a670d79-da94-415a-bb66-78d13b07bb7c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:32.638980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a670d79-da94-415a-bb66-78d13b07bb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:32.716423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5a670d79-da94-415a-bb66-78d13b07bb7c None None] MFA Rules not processed for user `780b62fec93b4ec58e2b4cb92e229771`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:32.717793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5a670d79-da94-415a-bb66-78d13b07bb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:32.773633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:32.822107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 661/1316] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:32 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:32.863808 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783b73e-78c4-464d-b7f4-f88b25fa83e3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:32.866585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6783b73e-78c4-464d-b7f4-f88b25fa83e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:32.879641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:32.981128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783b73e-78c4-464d-b7f4-f88b25fa83e3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:33.068875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877640d-3058-4b42-9129-07942d7c75e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:33.069537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877640d-3058-4b42-9129-07942d7c75e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:33.069863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877640d-3058-4b42-9129-07942d7c75e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:33.070124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877640d-3058-4b42-9129-07942d7c75e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:33.071766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9877640d-3058-4b42-9129-07942d7c75e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:33.143528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:33.194219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877640d-3058-4b42-9129-07942d7c75e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:33.195713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877640d-3058-4b42-9129-07942d7c75e0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:33.217591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 656/1317] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:32 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:35.892282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9077a91-6128-4a43-a35b-1abed14b8989 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:35.892834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9077a91-6128-4a43-a35b-1abed14b8989 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:35.892834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9077a91-6128-4a43-a35b-1abed14b8989 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:35.894172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 662/1318] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:35.899524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90ad2-68e8-4ba9-b95c-da2e9ca50af6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:35.899970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90ad2-68e8-4ba9-b95c-da2e9ca50af6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:35.900062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a90ad2-68e8-4ba9-b95c-da2e9ca50af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:35.970461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a3a90ad2-68e8-4ba9-b95c-da2e9ca50af6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:35.971699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a3a90ad2-68e8-4ba9-b95c-da2e9ca50af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:36.094412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 657/1319] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:36.146425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc65b55-6c9b-4aa7-9778-fe70813686e4 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:36.146694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc65b55-6c9b-4aa7-9778-fe70813686e4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:36.146869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc65b55-6c9b-4aa7-9778-fe70813686e4 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:36.148305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 663/1320] 10.222.0.71 () {54 vars in 965 bytes} [Mon Jul 18 14:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:36.155816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a39ec0-3aac-4244-b231-25544001dc62 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:36.156167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a39ec0-3aac-4244-b231-25544001dc62 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:36.156581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a39ec0-3aac-4244-b231-25544001dc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:36.182988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:36.251280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-83a39ec0-3aac-4244-b231-25544001dc62 None None] MFA Rules not processed for user `1f1fe084c3164e50bf5d904350925f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:36.253833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-83a39ec0-3aac-4244-b231-25544001dc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:36.343881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 658/1321] 10.222.0.71 () {58 vars in 1062 bytes} [Mon Jul 18 14:10:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:36.353082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc65b55-6c9b-4aa7-9778-fe70813686e4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:36.354677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5cc65b55-6c9b-4aa7-9778-fe70813686e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:36.484033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc65b55-6c9b-4aa7-9778-fe70813686e4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:36.570979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97cfbf-19b2-4782-afe4-4990dd67376c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:36.572098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cfbf-19b2-4782-afe4-4990dd67376c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:36.572569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cfbf-19b2-4782-afe4-4990dd67376c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:36.573022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97cfbf-19b2-4782-afe4-4990dd67376c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:36.575790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ba97cfbf-19b2-4782-afe4-4990dd67376c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:36.589595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:36.706083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97cfbf-19b2-4782-afe4-4990dd67376c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:36.707542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97cfbf-19b2-4782-afe4-4990dd67376c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:36.725932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 664/1322] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:40.163069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd7f17-f377-478e-87e8-4cf7846c3fdc None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:40.163687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd7f17-f377-478e-87e8-4cf7846c3fdc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:40.163787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd7f17-f377-478e-87e8-4cf7846c3fdc None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:40.165600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 659/1323] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:40.172251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d4245-430d-4a3e-95e4-8017d2d4302f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:40.172823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d4245-430d-4a3e-95e4-8017d2d4302f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:40.172823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d4245-430d-4a3e-95e4-8017d2d4302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:40.265606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-755d4245-430d-4a3e-95e4-8017d2d4302f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:40.267057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-755d4245-430d-4a3e-95e4-8017d2d4302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:40.340784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:40.388165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 665/1324] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:40.426663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd7f17-f377-478e-87e8-4cf7846c3fdc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:40.428493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-edbd7f17-f377-478e-87e8-4cf7846c3fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:40.560745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd7f17-f377-478e-87e8-4cf7846c3fdc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:40.573606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:40.638977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae2cbe-fa6e-496d-871a-4125ac096529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:40.639859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae2cbe-fa6e-496d-871a-4125ac096529 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:40.640442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae2cbe-fa6e-496d-871a-4125ac096529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:40.640719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae2cbe-fa6e-496d-871a-4125ac096529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:40.642416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-84ae2cbe-fa6e-496d-871a-4125ac096529 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:40.766762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae2cbe-fa6e-496d-871a-4125ac096529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:40.768005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae2cbe-fa6e-496d-871a-4125ac096529 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:40.787559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 660/1325] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:42.905384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc5e5-a126-44ec-bd00-54257f2bb8ec None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:42.905977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc5e5-a126-44ec-bd00-54257f2bb8ec None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:42.906284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fc5e5-a126-44ec-bd00-54257f2bb8ec None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:42.913935 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251470b4-c66b-4f45-91dc-df74abe47223 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:42.914716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251470b4-c66b-4f45-91dc-df74abe47223 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:42.914716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251470b4-c66b-4f45-91dc-df74abe47223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:42.917937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 666/1326] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:42.969768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:42.992645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-251470b4-c66b-4f45-91dc-df74abe47223 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:42.994119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-251470b4-c66b-4f45-91dc-df74abe47223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:43.126468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 661/1327] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:43.164588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fc5e5-a126-44ec-bd00-54257f2bb8ec None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:43.166195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-362fc5e5-a126-44ec-bd00-54257f2bb8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:43.285186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fc5e5-a126-44ec-bd00-54257f2bb8ec None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:43.383076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84b396-0ff5-479b-b766-e74ed077d75a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:43.383735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84b396-0ff5-479b-b766-e74ed077d75a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:43.384018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84b396-0ff5-479b-b766-e74ed077d75a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:43.384295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84b396-0ff5-479b-b766-e74ed077d75a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:43.386000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cf84b396-0ff5-479b-b766-e74ed077d75a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:43.420333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:43.534806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84b396-0ff5-479b-b766-e74ed077d75a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:43.539702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84b396-0ff5-479b-b766-e74ed077d75a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:43.562669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 667/1328] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:45.944772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bfc2-0576-4269-bd7e-8eeba823b8c4 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:45.945540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bfc2-0576-4269-bd7e-8eeba823b8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:45.945540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88bfc2-0576-4269-bd7e-8eeba823b8c4 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:45.946757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 662/1329] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:45.952531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096009a-4865-4bb6-bb52-294def1a9ee4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:45.953156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096009a-4865-4bb6-bb52-294def1a9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:45.953156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096009a-4865-4bb6-bb52-294def1a9ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:46.026755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a096009a-4865-4bb6-bb52-294def1a9ee4 None None] MFA Rules not processed for user `780b62fec93b4ec58e2b4cb92e229771`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:46.027885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a096009a-4865-4bb6-bb52-294def1a9ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:46.107398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:46.151647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 668/1330] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:46.194264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88bfc2-0576-4269-bd7e-8eeba823b8c4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:46.195842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7e88bfc2-0576-4269-bd7e-8eeba823b8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:46.320071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88bfc2-0576-4269-bd7e-8eeba823b8c4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:46.345005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:46.417173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c6be8-cd77-4a38-8e64-0eb755897365 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:46.418012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6be8-cd77-4a38-8e64-0eb755897365 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:46.418308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6be8-cd77-4a38-8e64-0eb755897365 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:46.418585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c6be8-cd77-4a38-8e64-0eb755897365 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:46.420727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-207c6be8-cd77-4a38-8e64-0eb755897365 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:46.551453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c6be8-cd77-4a38-8e64-0eb755897365 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:46.552825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c6be8-cd77-4a38-8e64-0eb755897365 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:46.573419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 663/1331] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:49.008992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6f3c0-b024-464c-b737-8ccfbe76fa1f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.009837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6f3c0-b024-464c-b737-8ccfbe76fa1f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.009837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6f3c0-b024-464c-b737-8ccfbe76fa1f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.011371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 669/1332] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:49.017924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c289dd-26fb-4a51-b7f9-7475563fd50f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.018393 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c289dd-26fb-4a51-b7f9-7475563fd50f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.018484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c289dd-26fb-4a51-b7f9-7475563fd50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.086919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:49.090343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-32c289dd-26fb-4a51-b7f9-7475563fd50f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:49.092879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-32c289dd-26fb-4a51-b7f9-7475563fd50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:49.194384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 664/1333] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:49.232803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6f3c0-b024-464c-b737-8ccfbe76fa1f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:49.234336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f8b6f3c0-b024-464c-b737-8ccfbe76fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:49.367677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6f3c0-b024-464c-b737-8ccfbe76fa1f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:49.449967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b466db77-acb1-42d7-ae66-4f0c725f6d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:49.450627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466db77-acb1-42d7-ae66-4f0c725f6d84 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:49.450910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466db77-acb1-42d7-ae66-4f0c725f6d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:49.451181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b466db77-acb1-42d7-ae66-4f0c725f6d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:49.453640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b466db77-acb1-42d7-ae66-4f0c725f6d84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:49.484189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:49.575083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466db77-acb1-42d7-ae66-4f0c725f6d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:49.577089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b466db77-acb1-42d7-ae66-4f0c725f6d84 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:49.596952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 670/1334] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:10:51.788361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e349a-83e5-4aa1-9eaa-5a762fb0bc16 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:51.788906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e349a-83e5-4aa1-9eaa-5a762fb0bc16 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:51.788906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e349a-83e5-4aa1-9eaa-5a762fb0bc16 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:51.790434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 665/1335] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:51.796073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999e724-ebd8-49b3-9784-d9bbd72d7e45 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:51.796557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999e724-ebd8-49b3-9784-d9bbd72d7e45 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:51.796651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6999e724-ebd8-49b3-9784-d9bbd72d7e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:51.868515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6999e724-ebd8-49b3-9784-d9bbd72d7e45 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:51.869789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6999e724-ebd8-49b3-9784-d9bbd72d7e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:51.949321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:51.972579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 671/1336] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:10:52.014084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e349a-83e5-4aa1-9eaa-5a762fb0bc16 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:52.015625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-aa9e349a-83e5-4aa1-9eaa-5a762fb0bc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:52.128462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e349a-83e5-4aa1-9eaa-5a762fb0bc16 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:52.160353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:52.208825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0dc77-1fe5-4bb1-8002-aeb2406b4278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:52.209389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0dc77-1fe5-4bb1-8002-aeb2406b4278 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:52.209704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0dc77-1fe5-4bb1-8002-aeb2406b4278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:52.209954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0dc77-1fe5-4bb1-8002-aeb2406b4278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:52.211611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-90f0dc77-1fe5-4bb1-8002-aeb2406b4278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:52.329854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0dc77-1fe5-4bb1-8002-aeb2406b4278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:52.331313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0dc77-1fe5-4bb1-8002-aeb2406b4278 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:52.350269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 666/1337] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:54.756651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9fe2-7bb0-4527-b863-ede9e9cff0f4 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:54.757297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9fe2-7bb0-4527-b863-ede9e9cff0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:54.757297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9fe2-7bb0-4527-b863-ede9e9cff0f4 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:54.758776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 672/1338] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:54.765025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc64eba-af4f-4745-a250-0d02a683c1d1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:54.765615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc64eba-af4f-4745-a250-0d02a683c1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:54.765615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc64eba-af4f-4745-a250-0d02a683c1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:54.837714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0cc64eba-af4f-4745-a250-0d02a683c1d1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:54.839012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0cc64eba-af4f-4745-a250-0d02a683c1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:54.848693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:54.957143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 667/1339] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:10:54.994528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9fe2-7bb0-4527-b863-ede9e9cff0f4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:54.996090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ea5c9fe2-7bb0-4527-b863-ede9e9cff0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:55.127635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9fe2-7bb0-4527-b863-ede9e9cff0f4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:10:55.214423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab18e42-5ca0-48a0-892c-75970439b54d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:10:55.215021 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18e42-5ca0-48a0-892c-75970439b54d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:55.215303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18e42-5ca0-48a0-892c-75970439b54d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:55.215573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab18e42-5ca0-48a0-892c-75970439b54d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:55.217433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cab18e42-5ca0-48a0-892c-75970439b54d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:55.265495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:55.349213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab18e42-5ca0-48a0-892c-75970439b54d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:10:55.350703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab18e42-5ca0-48a0-892c-75970439b54d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:10:55.374172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 673/1340] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:10:59.619001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded04a1-1594-430d-923c-640b03f54aac None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:59.619001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded04a1-1594-430d-923c-640b03f54aac None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:59.619001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded04a1-1594-430d-923c-640b03f54aac None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:59.620775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 668/1341] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:10:59.626668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2094-e5d3-4bff-83f7-797a9d67c99d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:10:59.627360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2094-e5d3-4bff-83f7-797a9d67c99d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:10:59.627360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2094-e5d3-4bff-83f7-797a9d67c99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:10:59.705476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-98ac2094-e5d3-4bff-83f7-797a9d67c99d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:10:59.706730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-98ac2094-e5d3-4bff-83f7-797a9d67c99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:59.805591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:10:59.819534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 674/1342] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:10:59.860424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded04a1-1594-430d-923c-640b03f54aac None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:10:59.862227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bded04a1-1594-430d-923c-640b03f54aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:10:59.971610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded04a1-1594-430d-923c-640b03f54aac None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:00.010279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:00.049091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9be343-a854-4946-b91c-5a018fe2e463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:00.049683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9be343-a854-4946-b91c-5a018fe2e463 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:00.049980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9be343-a854-4946-b91c-5a018fe2e463 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:00.050257 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9be343-a854-4946-b91c-5a018fe2e463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:00.051920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ab9be343-a854-4946-b91c-5a018fe2e463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:00.170596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9be343-a854-4946-b91c-5a018fe2e463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:00.172055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9be343-a854-4946-b91c-5a018fe2e463 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:00.192762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 669/1343] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:10:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:02.946147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7ac2d-82b6-4250-8077-37cc53c20bd7 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:02.946698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7ac2d-82b6-4250-8077-37cc53c20bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:02.946698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7ac2d-82b6-4250-8077-37cc53c20bd7 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:02.949755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 675/1344] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:02.954312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02a34-b8b7-4fd8-b2ae-f8a06ff30630 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:02.954869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02a34-b8b7-4fd8-b2ae-f8a06ff30630 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:02.954869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e02a34-b8b7-4fd8-b2ae-f8a06ff30630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:03.033322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-25e02a34-b8b7-4fd8-b2ae-f8a06ff30630 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:03.034615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25e02a34-b8b7-4fd8-b2ae-f8a06ff30630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:03.055022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:03.153633 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 670/1345] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:03.192608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7ac2d-82b6-4250-8077-37cc53c20bd7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:03.194250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-22a7ac2d-82b6-4250-8077-37cc53c20bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:03.329348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7ac2d-82b6-4250-8077-37cc53c20bd7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:03.422522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a705ae5-d6e2-4d8a-ba72-588932d76841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:03.423186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705ae5-d6e2-4d8a-ba72-588932d76841 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:03.423480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705ae5-d6e2-4d8a-ba72-588932d76841 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:03.423756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a705ae5-d6e2-4d8a-ba72-588932d76841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:03.425468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4a705ae5-d6e2-4d8a-ba72-588932d76841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:03.475049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:03.558372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a705ae5-d6e2-4d8a-ba72-588932d76841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:03.561265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a705ae5-d6e2-4d8a-ba72-588932d76841 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:03.584391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 676/1346] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:07.817181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c30fc-6482-4f0a-9918-38ee94e5b289 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:07.817800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c30fc-6482-4f0a-9918-38ee94e5b289 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:07.817800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c30fc-6482-4f0a-9918-38ee94e5b289 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:07.819189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 671/1347] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:07.824591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccc4a1-44e2-453a-998a-765e4f926f3d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:07.825094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccc4a1-44e2-453a-998a-765e4f926f3d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:07.825245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccc4a1-44e2-453a-998a-765e4f926f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:07.898860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f1ccc4a1-44e2-453a-998a-765e4f926f3d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:07.900015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f1ccc4a1-44e2-453a-998a-765e4f926f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:08.037085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:08.039938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 677/1348] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:08.078539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c30fc-6482-4f0a-9918-38ee94e5b289 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:08.080092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2b9c30fc-6482-4f0a-9918-38ee94e5b289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:08.205589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c30fc-6482-4f0a-9918-38ee94e5b289 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:08.268148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:08.302024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9093af1-4663-4c29-bf6b-873be8c70c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:08.302693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9093af1-4663-4c29-bf6b-873be8c70c0a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:08.302973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9093af1-4663-4c29-bf6b-873be8c70c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:08.303248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9093af1-4663-4c29-bf6b-873be8c70c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:08.304963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a9093af1-4663-4c29-bf6b-873be8c70c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:08.449346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9093af1-4663-4c29-bf6b-873be8c70c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:08.451145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9093af1-4663-4c29-bf6b-873be8c70c0a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:08.474475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 672/1349] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:13.698205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10da819-69b7-4be3-91bf-b67d77627f70 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:13.698646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10da819-69b7-4be3-91bf-b67d77627f70 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:13.698870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10da819-69b7-4be3-91bf-b67d77627f70 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:13.700619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 678/1350] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:11:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:13.707644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62799e-6064-488d-b8cb-7185aa29e57f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:13.707644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62799e-6064-488d-b8cb-7185aa29e57f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:13.708291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62799e-6064-488d-b8cb-7185aa29e57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:13.847044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6c62799e-6064-488d-b8cb-7185aa29e57f None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:13.848169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c62799e-6064-488d-b8cb-7185aa29e57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:13.870279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:14.018007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 673/1351] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:11:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:18.756596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b112ec-1b55-4a2b-90e4-333d4f851cb2 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:18.757480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b112ec-1b55-4a2b-90e4-333d4f851cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:18.757965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b112ec-1b55-4a2b-90e4-333d4f851cb2 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:18.759683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 679/1352] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:11:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:18.767319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb8c5-76f5-43e8-9018-e11edfb1a782 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:18.767898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb8c5-76f5-43e8-9018-e11edfb1a782 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:18.768057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eb8c5-76f5-43e8-9018-e11edfb1a782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:18.862608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9e8eb8c5-76f5-43e8-9018-e11edfb1a782 None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:18.864316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9e8eb8c5-76f5-43e8-9018-e11edfb1a782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:18.990878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:19.041727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 674/1353] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:11:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:20.619319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382379e4-3b32-41b3-b197-ef02457c4e0f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:20.619939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382379e4-3b32-41b3-b197-ef02457c4e0f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:20.619939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382379e4-3b32-41b3-b197-ef02457c4e0f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:20.621231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 680/1354] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:20.626838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b4a9-1ecc-4049-8c15-2f468efc0604 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:20.627529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b4a9-1ecc-4049-8c15-2f468efc0604 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:20.627529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1b4a9-1ecc-4049-8c15-2f468efc0604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:20.713192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ada1b4a9-1ecc-4049-8c15-2f468efc0604 None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:20.715276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ada1b4a9-1ecc-4049-8c15-2f468efc0604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:20.830980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 675/1355] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:11:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:24.345514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f05a-c799-4f61-bfd9-58f15b265403 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:24.346404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f05a-c799-4f61-bfd9-58f15b265403 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:24.346831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8f05a-c799-4f61-bfd9-58f15b265403 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:24.348344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 681/1356] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:24.354024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976768c-7375-4d5c-a6e4-6052cc4ec9dc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:24.354470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976768c-7375-4d5c-a6e4-6052cc4ec9dc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:24.354586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976768c-7375-4d5c-a6e4-6052cc4ec9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:24.391912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:24.425414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0976768c-7375-4d5c-a6e4-6052cc4ec9dc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:24.426673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0976768c-7375-4d5c-a6e4-6052cc4ec9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:24.585525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 676/1357] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:24.670373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e64997-6484-438c-bef5-2810e447856c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:24.671504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e64997-6484-438c-bef5-2810e447856c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:24.671784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e64997-6484-438c-bef5-2810e447856c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:24.673273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 682/1358] 10.222.0.71 () {54 vars in 962 bytes} [Mon Jul 18 14:11:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:24.692252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7067-b4c1-424d-bcfc-0b79951971b5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:24.698444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7067-b4c1-424d-bcfc-0b79951971b5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:24.699136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db7067-b4c1-424d-bcfc-0b79951971b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:24.812359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-28db7067-b4c1-424d-bcfc-0b79951971b5 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:24.813798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-28db7067-b4c1-424d-bcfc-0b79951971b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:24.869107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:24.947668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 677/1359] 10.222.0.71 () {58 vars in 1059 bytes} [Mon Jul 18 14:11:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:24.956949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e64997-6484-438c-bef5-2810e447856c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:24.958488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b4e64997-6484-438c-bef5-2810e447856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:25.134630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e64997-6484-438c-bef5-2810e447856c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:25.220179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:25.271052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462bf24-b053-4eb3-9c4b-2f9bf1e5eb58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:25.271733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462bf24-b053-4eb3-9c4b-2f9bf1e5eb58 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:25.272018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462bf24-b053-4eb3-9c4b-2f9bf1e5eb58 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:25.272299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462bf24-b053-4eb3-9c4b-2f9bf1e5eb58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:25.274010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c462bf24-b053-4eb3-9c4b-2f9bf1e5eb58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:25.367651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462bf24-b053-4eb3-9c4b-2f9bf1e5eb58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:25.369344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462bf24-b053-4eb3-9c4b-2f9bf1e5eb58 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:25.388976 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 683/1360] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:27.620368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44f683-d675-48fa-b9ce-375e93f515d8 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:27.620844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44f683-d675-48fa-b9ce-375e93f515d8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:27.620936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d44f683-d675-48fa-b9ce-375e93f515d8 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:27.622805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 678/1361] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:27.629874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db75c42-4080-4446-8dec-7ab6afe18839 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:27.631137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db75c42-4080-4446-8dec-7ab6afe18839 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:27.631651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db75c42-4080-4446-8dec-7ab6afe18839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:27.705441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2db75c42-4080-4446-8dec-7ab6afe18839 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:27.706962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2db75c42-4080-4446-8dec-7ab6afe18839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:27.748579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:27.828590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 684/1362] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:27.874547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44f683-d675-48fa-b9ce-375e93f515d8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:27.876072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7d44f683-d675-48fa-b9ce-375e93f515d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:28.047965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d44f683-d675-48fa-b9ce-375e93f515d8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:28.059152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:28.222152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059153b5-ad6d-4a68-abba-873ea1347541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:28.222822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059153b5-ad6d-4a68-abba-873ea1347541 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:28.223107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059153b5-ad6d-4a68-abba-873ea1347541 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:28.223378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059153b5-ad6d-4a68-abba-873ea1347541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:28.225096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-059153b5-ad6d-4a68-abba-873ea1347541 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:28.338094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059153b5-ad6d-4a68-abba-873ea1347541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:28.339847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059153b5-ad6d-4a68-abba-873ea1347541 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:28.359763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 679/1363] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:30.716903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc62f4-6bc5-4e72-90b1-d9580c8eac71 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:30.717432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc62f4-6bc5-4e72-90b1-d9580c8eac71 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:30.717432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc62f4-6bc5-4e72-90b1-d9580c8eac71 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:30.718880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 685/1364] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:30.724772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712100-61f7-4a1c-823e-387b65c7bbfc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:30.725208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712100-61f7-4a1c-823e-387b65c7bbfc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:30.725356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712100-61f7-4a1c-823e-387b65c7bbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:30.742275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:30.806330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d9712100-61f7-4a1c-823e-387b65c7bbfc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:30.808159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d9712100-61f7-4a1c-823e-387b65c7bbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:30.924099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 680/1365] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:30.969864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08541f3-4d2e-4534-ab4d-2069b4971066 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:30.970477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08541f3-4d2e-4534-ab4d-2069b4971066 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:30.970956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08541f3-4d2e-4534-ab4d-2069b4971066 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:30.972640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 686/1366] 10.222.0.71 () {54 vars in 962 bytes} [Mon Jul 18 14:11:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:30.979429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3daf2-c8ea-4126-bba8-68b29906583b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:30.979429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3daf2-c8ea-4126-bba8-68b29906583b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:30.979429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3daf2-c8ea-4126-bba8-68b29906583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:31.075140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-32f3daf2-c8ea-4126-bba8-68b29906583b None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:31.076568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-32f3daf2-c8ea-4126-bba8-68b29906583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:31.123336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:31.245841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 681/1367] 10.222.0.71 () {58 vars in 1059 bytes} [Mon Jul 18 14:11:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:31.255540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08541f3-4d2e-4534-ab4d-2069b4971066 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:31.257286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d08541f3-4d2e-4534-ab4d-2069b4971066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:31.424654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08541f3-4d2e-4534-ab4d-2069b4971066 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:31.447674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:31.554043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac2377-42e7-4618-93b3-9fb06e9da65f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:31.554615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2377-42e7-4618-93b3-9fb06e9da65f service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:31.554899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2377-42e7-4618-93b3-9fb06e9da65f service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:31.555173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac2377-42e7-4618-93b3-9fb06e9da65f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:31.556850 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3bac2377-42e7-4618-93b3-9fb06e9da65f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:31.666234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac2377-42e7-4618-93b3-9fb06e9da65f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:31.667875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac2377-42e7-4618-93b3-9fb06e9da65f service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:31.688341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 687/1368] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:33.035280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c4f3-4917-4bff-9d43-8de4b42b39b5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:33.036114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c4f3-4917-4bff-9d43-8de4b42b39b5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:33.036114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c4f3-4917-4bff-9d43-8de4b42b39b5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:33.037411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 682/1369] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:11:33.042948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73172dae-0829-4543-bd87-874b2db2e388 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:33.044225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73172dae-0829-4543-bd87-874b2db2e388 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:33.044225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73172dae-0829-4543-bd87-874b2db2e388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:33.080805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:33.126224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-73172dae-0829-4543-bd87-874b2db2e388 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:33.127496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-73172dae-0829-4543-bd87-874b2db2e388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:33.252437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 688/1370] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:33.591374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644923-f444-4dc9-9589-322aeec03d3d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:33.591774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644923-f444-4dc9-9589-322aeec03d3d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:33.591967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644923-f444-4dc9-9589-322aeec03d3d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:33.593924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 683/1371] 10.222.0.71 () {54 vars in 964 bytes} [Mon Jul 18 14:11:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:33.599131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6ca1-5193-4701-b31d-30c555c25cb5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:33.599428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6ca1-5193-4701-b31d-30c555c25cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:33.599824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59c6ca1-5193-4701-b31d-30c555c25cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:33.730254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c59c6ca1-5193-4701-b31d-30c555c25cb5 None None] MFA Rules not processed for user `812ba8119a5a4297b409525861ec8118`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:33.731995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c59c6ca1-5193-4701-b31d-30c555c25cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:33.809195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:33.853138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 689/1372] 10.222.0.71 () {58 vars in 1061 bytes} [Mon Jul 18 14:11:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:33.858151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebb706-2d4b-4f89-97fb-01a8d47353f0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:33.858444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebb706-2d4b-4f89-97fb-01a8d47353f0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:33.858724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebb706-2d4b-4f89-97fb-01a8d47353f0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:33.862129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c6ca1-5193-4701-b31d-30c555c25cb5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:33.877921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 684/1373] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:33.877921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18962e-a4e8-4241-8b56-c08a410398bd None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:33.877921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18962e-a4e8-4241-8b56-c08a410398bd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:33.877921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18962e-a4e8-4241-8b56-c08a410398bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:33.879199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59c6ca1-5193-4701-b31d-30c555c25cb5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:33.971175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bb18962e-a4e8-4241-8b56-c08a410398bd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:33.972415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bb18962e-a4e8-4241-8b56-c08a410398bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:33.994742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686aab7-4e41-4ee0-a33d-67472682e0e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:33.995419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686aab7-4e41-4ee0-a33d-67472682e0e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:33.995709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686aab7-4e41-4ee0-a33d-67472682e0e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:33.996141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686aab7-4e41-4ee0-a33d-67472682e0e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:33.998717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e686aab7-4e41-4ee0-a33d-67472682e0e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:34.097768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:34.109788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 685/1374] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:34.153633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686aab7-4e41-4ee0-a33d-67472682e0e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:34.154826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686aab7-4e41-4ee0-a33d-67472682e0e9 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:34.160152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18962e-a4e8-4241-8b56-c08a410398bd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:34.161509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bb18962e-a4e8-4241-8b56-c08a410398bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:34.176419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:34.192847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 690/1375] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:11:34.299713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18962e-a4e8-4241-8b56-c08a410398bd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:34.422034 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:34.435217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a0f43-113e-4467-b27a-ee65e504b6d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:34.435217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a0f43-113e-4467-b27a-ee65e504b6d0 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:34.435217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a0f43-113e-4467-b27a-ee65e504b6d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:34.435217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a0f43-113e-4467-b27a-ee65e504b6d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:34.438055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-499a0f43-113e-4467-b27a-ee65e504b6d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:34.613811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a0f43-113e-4467-b27a-ee65e504b6d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:34.616049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499a0f43-113e-4467-b27a-ee65e504b6d0 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:34.646017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 686/1376] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:36.808265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c809f1-b83a-43c4-a71e-a8945a20ef16 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:36.808819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c809f1-b83a-43c4-a71e-a8945a20ef16 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:36.809044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c809f1-b83a-43c4-a71e-a8945a20ef16 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:36.810817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 691/1377] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:36.816569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a9277-6aea-4e44-ae28-c1b8dda42ed1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:36.817080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a9277-6aea-4e44-ae28-c1b8dda42ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:36.817431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a9277-6aea-4e44-ae28-c1b8dda42ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:36.891695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-920a9277-6aea-4e44-ae28-c1b8dda42ed1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:36.892905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-920a9277-6aea-4e44-ae28-c1b8dda42ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:36.963996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:37.018787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 687/1378] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:37.065324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c809f1-b83a-43c4-a71e-a8945a20ef16 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:37.067331 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-32c809f1-b83a-43c4-a71e-a8945a20ef16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:37.233056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c809f1-b83a-43c4-a71e-a8945a20ef16 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:37.324992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:37.389243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba88c2-d076-4861-b252-cd41068c8c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:37.390253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba88c2-d076-4861-b252-cd41068c8c06 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:37.390533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba88c2-d076-4861-b252-cd41068c8c06 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:37.390806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba88c2-d076-4861-b252-cd41068c8c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:37.392519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-95ba88c2-d076-4861-b252-cd41068c8c06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:37.514397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba88c2-d076-4861-b252-cd41068c8c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:37.516001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba88c2-d076-4861-b252-cd41068c8c06 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:37.541324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 692/1379] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:39.672664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617752-5380-43a3-8376-1dc20592530a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:39.673311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617752-5380-43a3-8376-1dc20592530a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:39.673311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617752-5380-43a3-8376-1dc20592530a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:39.674678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 688/1380] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:39.680657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627739b-2260-4234-9004-62dbf8ceb0cb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:39.681195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627739b-2260-4234-9004-62dbf8ceb0cb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:39.681518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627739b-2260-4234-9004-62dbf8ceb0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:39.756686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8627739b-2260-4234-9004-62dbf8ceb0cb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:39.757961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8627739b-2260-4234-9004-62dbf8ceb0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:39.796845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:39.884468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 693/1381] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:39.924463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30617752-5380-43a3-8376-1dc20592530a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:39.926112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-30617752-5380-43a3-8376-1dc20592530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:40.109091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30617752-5380-43a3-8376-1dc20592530a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:40.166764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:40.246754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5ad33-db0f-4ff8-9c65-a007a5613965 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:40.247741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5ad33-db0f-4ff8-9c65-a007a5613965 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:40.248155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5ad33-db0f-4ff8-9c65-a007a5613965 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:40.248650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5ad33-db0f-4ff8-9c65-a007a5613965 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:40.251067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bdd5ad33-db0f-4ff8-9c65-a007a5613965 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:40.368576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5ad33-db0f-4ff8-9c65-a007a5613965 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:40.370315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5ad33-db0f-4ff8-9c65-a007a5613965 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:40.392637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 689/1382] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:42.627864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e844-e05b-44b0-b1dd-d54a2781bb17 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:42.628705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e844-e05b-44b0-b1dd-d54a2781bb17 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:42.628879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16e844-e05b-44b0-b1dd-d54a2781bb17 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:42.631242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 694/1383] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:42.638987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03027e37-236c-4639-98c2-dd08412ad3bc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:42.639685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03027e37-236c-4639-98c2-dd08412ad3bc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:42.639794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03027e37-236c-4639-98c2-dd08412ad3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:42.743081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-03027e37-236c-4639-98c2-dd08412ad3bc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:42.745154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-03027e37-236c-4639-98c2-dd08412ad3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:42.749526 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:42.876111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 690/1384] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:42.923090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e844-e05b-44b0-b1dd-d54a2781bb17 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:42.924832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6c16e844-e05b-44b0-b1dd-d54a2781bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:43.103018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16e844-e05b-44b0-b1dd-d54a2781bb17 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:43.120396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:43.267177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f56cdf9-d32b-4f7b-9cef-a3971a5677d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:43.268442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56cdf9-d32b-4f7b-9cef-a3971a5677d8 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:43.268979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56cdf9-d32b-4f7b-9cef-a3971a5677d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:43.269519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56cdf9-d32b-4f7b-9cef-a3971a5677d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:43.271968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0f56cdf9-d32b-4f7b-9cef-a3971a5677d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:43.411411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f56cdf9-d32b-4f7b-9cef-a3971a5677d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:43.413171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f56cdf9-d32b-4f7b-9cef-a3971a5677d8 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:43.437208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 695/1385] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:45.567357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364004d8-0126-4bb4-8f1f-1e860b55b525 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:45.568157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364004d8-0126-4bb4-8f1f-1e860b55b525 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:45.568157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364004d8-0126-4bb4-8f1f-1e860b55b525 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:45.570166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 691/1386] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:45.576182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f642f-1e5b-49eb-a232-820629c4f687 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:45.577138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f642f-1e5b-49eb-a232-820629c4f687 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:45.577436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f642f-1e5b-49eb-a232-820629c4f687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:45.606218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:45.677586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f17f642f-1e5b-49eb-a232-820629c4f687 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:45.679359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f17f642f-1e5b-49eb-a232-820629c4f687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:45.818449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 696/1387] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:45.875227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364004d8-0126-4bb4-8f1f-1e860b55b525 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:45.877389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-364004d8-0126-4bb4-8f1f-1e860b55b525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:46.019746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:46.038301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364004d8-0126-4bb4-8f1f-1e860b55b525 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:46.181728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea67b8-d3fc-4173-a2c2-c45860ce3ac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:46.182983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea67b8-d3fc-4173-a2c2-c45860ce3ac9 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:46.183943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea67b8-d3fc-4173-a2c2-c45860ce3ac9 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:46.184553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea67b8-d3fc-4173-a2c2-c45860ce3ac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:46.186943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-67ea67b8-d3fc-4173-a2c2-c45860ce3ac9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:46.305362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea67b8-d3fc-4173-a2c2-c45860ce3ac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:46.307024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea67b8-d3fc-4173-a2c2-c45860ce3ac9 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:46.314099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:46.331133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 692/1388] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:48.719034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7816c15-893e-47dc-9217-23cc4d4170b8 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:48.719597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7816c15-893e-47dc-9217-23cc4d4170b8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:48.719597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7816c15-893e-47dc-9217-23cc4d4170b8 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:48.720986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 697/1389] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:48.726880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf81f20-9939-41d2-907c-bf9ecab69e2f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:48.727385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf81f20-9939-41d2-907c-bf9ecab69e2f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:48.727385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf81f20-9939-41d2-907c-bf9ecab69e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:48.798369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-baf81f20-9939-41d2-907c-bf9ecab69e2f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:48.799644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-baf81f20-9939-41d2-907c-bf9ecab69e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:48.904978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 693/1390] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:48.949808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7816c15-893e-47dc-9217-23cc4d4170b8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:48.951377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d7816c15-893e-47dc-9217-23cc4d4170b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:49.052383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:49.110894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7816c15-893e-47dc-9217-23cc4d4170b8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:49.249346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554696e-2fe6-4eb9-9286-3d69f2b8b9b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:49.249968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554696e-2fe6-4eb9-9286-3d69f2b8b9b0 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:49.250259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554696e-2fe6-4eb9-9286-3d69f2b8b9b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:49.250537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554696e-2fe6-4eb9-9286-3d69f2b8b9b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:49.252327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b554696e-2fe6-4eb9-9286-3d69f2b8b9b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:49.341302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:49.375928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554696e-2fe6-4eb9-9286-3d69f2b8b9b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:49.377591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554696e-2fe6-4eb9-9286-3d69f2b8b9b0 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:49.400238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 698/1391] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:49.791827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424572b-d119-4a69-af97-db1a15ff56fd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:49.792437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424572b-d119-4a69-af97-db1a15ff56fd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:49.792643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424572b-d119-4a69-af97-db1a15ff56fd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:49.794296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 694/1392] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:49.801544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dbaf-1807-4370-8303-87996e1250f2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:49.801544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dbaf-1807-4370-8303-87996e1250f2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:49.801544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6dbaf-1807-4370-8303-87996e1250f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:49.875272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-42a6dbaf-1807-4370-8303-87996e1250f2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:49.876402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-42a6dbaf-1807-4370-8303-87996e1250f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:49.984740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 699/1393] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:50.044437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b1d19-c3f8-4c03-b5f2-37127c564289 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.044710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b1d19-c3f8-4c03-b5f2-37127c564289 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.045086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b1d19-c3f8-4c03-b5f2-37127c564289 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.046853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 695/1394] 10.222.0.71 () {54 vars in 961 bytes} [Mon Jul 18 14:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:50.052396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1081ed3-9596-4d58-9348-1ad5194ecc6f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.052749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1081ed3-9596-4d58-9348-1ad5194ecc6f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.052987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1081ed3-9596-4d58-9348-1ad5194ecc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.111525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:50.145758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e1081ed3-9596-4d58-9348-1ad5194ecc6f None None] MFA Rules not processed for user `d661615b2dd94d9db6183f580ab0fee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:50.147010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e1081ed3-9596-4d58-9348-1ad5194ecc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:50.232769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 700/1395] 10.222.0.71 () {58 vars in 1058 bytes} [Mon Jul 18 14:11:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:50.240871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b1d19-c3f8-4c03-b5f2-37127c564289 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:50.242652 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-256b1d19-c3f8-4c03-b5f2-37127c564289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:50.327026 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:50.363462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b1d19-c3f8-4c03-b5f2-37127c564289 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:50.454454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d43380-c408-406f-b106-cc63a7dbf542 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:50.455043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43380-c408-406f-b106-cc63a7dbf542 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:50.455327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43380-c408-406f-b106-cc63a7dbf542 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:50.455641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43380-c408-406f-b106-cc63a7dbf542 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:50.457438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e4d43380-c408-406f-b106-cc63a7dbf542 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:50.622706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d43380-c408-406f-b106-cc63a7dbf542 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:50.624184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d43380-c408-406f-b106-cc63a7dbf542 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:50.646445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:50.650971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 696/1396] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:51.496165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d76642-360d-4100-bff9-a260bfb969b3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.496861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d76642-360d-4100-bff9-a260bfb969b3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.496861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d76642-360d-4100-bff9-a260bfb969b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.555817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c510-a00a-4e93-a523-8701d1e5d64b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.556423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c510-a00a-4e93-a523-8701d1e5d64b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.556535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c510-a00a-4e93-a523-8701d1e5d64b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.558671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 697/1397] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:11:51.564473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb9cf3-5491-4b5c-87f5-ea266e20dda3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.564753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb9cf3-5491-4b5c-87f5-ea266e20dda3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.565032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb9cf3-5491-4b5c-87f5-ea266e20dda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.581959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-58d76642-360d-4100-bff9-a260bfb969b3 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:51.583254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-58d76642-360d-4100-bff9-a260bfb969b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:51.642469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-86fb9cf3-5491-4b5c-87f5-ea266e20dda3 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:51.644300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-86fb9cf3-5491-4b5c-87f5-ea266e20dda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:51.681483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:51.689722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 701/1398] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:11:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:51.744212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadf0a1-44b7-40b4-a8f4-4e905374ec5d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.744499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadf0a1-44b7-40b4-a8f4-4e905374ec5d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.744779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eadf0a1-44b7-40b4-a8f4-4e905374ec5d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.746757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 702/1399] 10.222.0.71 () {54 vars in 943 bytes} [Mon Jul 18 14:11:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:51.754047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c82eb6-301c-4418-b380-444a94c40e48 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:51.754328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c82eb6-301c-4418-b380-444a94c40e48 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:51.754603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c82eb6-301c-4418-b380-444a94c40e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:51.778497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 698/1400] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:11:51.826742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb9cf3-5491-4b5c-87f5-ea266e20dda3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:51.828100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-86fb9cf3-5491-4b5c-87f5-ea266e20dda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:51.842212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-31c82eb6-301c-4418-b380-444a94c40e48 None None] MFA Rules not processed for user `203e654f95bf45e683b0ccc4ba687bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:51.843448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-31c82eb6-301c-4418-b380-444a94c40e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:51.910541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:51.954736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb9cf3-5491-4b5c-87f5-ea266e20dda3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:51.967587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 703/1401] 10.222.0.71 () {58 vars in 1040 bytes} [Mon Jul 18 14:11:51 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:51.977141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c82eb6-301c-4418-b380-444a94c40e48 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:51.988398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c82eb6-301c-4418-b380-444a94c40e48 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:52.045362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:52.103605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdc469-3288-4e49-bd50-caa11e21a872 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:52.104265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdc469-3288-4e49-bd50-caa11e21a872 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:52.104545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdc469-3288-4e49-bd50-caa11e21a872 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:52.104829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdc469-3288-4e49-bd50-caa11e21a872 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:52.106585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5efdc469-3288-4e49-bd50-caa11e21a872 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:52.107201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5c8fc-c42c-4ceb-a753-d510b9665f7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:52.107867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5c8fc-c42c-4ceb-a753-d510b9665f7d service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:52.108157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5c8fc-c42c-4ceb-a753-d510b9665f7d service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:52.108430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5c8fc-c42c-4ceb-a753-d510b9665f7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:52.110172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a0b5c8fc-c42c-4ceb-a753-d510b9665f7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:52.189231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:52.201112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdc469-3288-4e49-bd50-caa11e21a872 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:52.202906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdc469-3288-4e49-bd50-caa11e21a872 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:52.212848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 704/1402] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:11:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 14:11:52.215736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5c8fc-c42c-4ceb-a753-d510b9665f7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:52.217265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5c8fc-c42c-4ceb-a753-d510b9665f7d service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:52.239365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 699/1403] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:54.352761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729581-78f1-4e55-99b6-9736e3e2c0e5 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:54.353540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729581-78f1-4e55-99b6-9736e3e2c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:54.353540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6729581-78f1-4e55-99b6-9736e3e2c0e5 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:54.354825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 705/1404] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:54.361222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804361-8631-4870-82ee-413ad3d67591 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:54.361828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804361-8631-4870-82ee-413ad3d67591 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:54.361970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1804361-8631-4870-82ee-413ad3d67591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:54.485552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f1804361-8631-4870-82ee-413ad3d67591 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:54.487577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f1804361-8631-4870-82ee-413ad3d67591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:54.651416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 700/1405] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:54.701950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6729581-78f1-4e55-99b6-9736e3e2c0e5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:54.703485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d6729581-78f1-4e55-99b6-9736e3e2c0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:54.811220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:54.909129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6729581-78f1-4e55-99b6-9736e3e2c0e5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:55.076230 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f6e46-f48f-4f7d-8549-1d4fc719ba89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:55.076908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f6e46-f48f-4f7d-8549-1d4fc719ba89 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:55.077332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f6e46-f48f-4f7d-8549-1d4fc719ba89 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:55.077625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f6e46-f48f-4f7d-8549-1d4fc719ba89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:55.079491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d45f6e46-f48f-4f7d-8549-1d4fc719ba89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:55.143886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:55.235083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f6e46-f48f-4f7d-8549-1d4fc719ba89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:55.237612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f6e46-f48f-4f7d-8549-1d4fc719ba89 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:55.268445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 706/1406] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:11:57.445439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e688ea-74e9-4306-b7a8-ba3602d9804b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:57.446022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e688ea-74e9-4306-b7a8-ba3602d9804b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:57.446022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e688ea-74e9-4306-b7a8-ba3602d9804b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:57.451118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 701/1407] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:11:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:11:57.453157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b9628-c75a-4354-a00e-818c8cb7f0ca None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:57.453642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b9628-c75a-4354-a00e-818c8cb7f0ca None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:57.453778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b9628-c75a-4354-a00e-818c8cb7f0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:57.531988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f94b9628-c75a-4354-a00e-818c8cb7f0ca None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:11:57.533341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f94b9628-c75a-4354-a00e-818c8cb7f0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:57.681228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 707/1408] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:11:57.733530 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e688ea-74e9-4306-b7a8-ba3602d9804b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:11:57.735106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-38e688ea-74e9-4306-b7a8-ba3602d9804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:57.805070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:57.936988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e688ea-74e9-4306-b7a8-ba3602d9804b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:11:58.108628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1f7ce-5b14-445d-a3d2-2a8b967daec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:11:58.109472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f7ce-5b14-445d-a3d2-2a8b967daec8 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:11:58.109893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f7ce-5b14-445d-a3d2-2a8b967daec8 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:11:58.110354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1f7ce-5b14-445d-a3d2-2a8b967daec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:11:58.113327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a1d1f7ce-5b14-445d-a3d2-2a8b967daec8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:11:58.134616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:11:58.287585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1f7ce-5b14-445d-a3d2-2a8b967daec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:11:58.291205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d1f7ce-5b14-445d-a3d2-2a8b967daec8 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:11:58.314826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 702/1409] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:11:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:07.679202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:08.312253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=39 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:15.464804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e569ddc-c3fe-4adf-a8d9-9deaefa6cd02 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:15.465488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e569ddc-c3fe-4adf-a8d9-9deaefa6cd02 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:15.465488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e569ddc-c3fe-4adf-a8d9-9deaefa6cd02 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:15.466990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 708/1410] 10.222.0.71 () {54 vars in 918 bytes} [Mon Jul 18 14:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:15.472815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0710f-3599-467d-a0bd-7264f732e739 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:15.473368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0710f-3599-467d-a0bd-7264f732e739 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:15.473481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da0710f-3599-467d-a0bd-7264f732e739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:15.560392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8da0710f-3599-467d-a0bd-7264f732e739 None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:15.561631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8da0710f-3599-467d-a0bd-7264f732e739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:15.682488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 703/1411] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Jul 18 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:15.697606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094ac52-877f-4d8a-806f-e8994188a7fc None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:15.697923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094ac52-877f-4d8a-806f-e8994188a7fc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:15.698204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094ac52-877f-4d8a-806f-e8994188a7fc None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:15.699662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 709/1412] 10.222.0.71 () {54 vars in 918 bytes} [Mon Jul 18 14:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:15.705130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df288-d04f-4cb0-8532-f58b714ab064 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:15.705529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df288-d04f-4cb0-8532-f58b714ab064 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:15.705983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1df288-d04f-4cb0-8532-f58b714ab064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:15.787002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:15.790270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5c1df288-d04f-4cb0-8532-f58b714ab064 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:15.792146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5c1df288-d04f-4cb0-8532-f58b714ab064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:15.939481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 704/1413] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Jul 18 14:12:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:18.608701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308864-b301-4b23-93d0-6f5ac4bd30d6 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:18.609310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308864-b301-4b23-93d0-6f5ac4bd30d6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:18.609310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308864-b301-4b23-93d0-6f5ac4bd30d6 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:18.610642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:18.617072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ec8b7-f246-4682-9d25-1c0edef0a086 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:18.617553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ec8b7-f246-4682-9d25-1c0edef0a086 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:18.617681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ec8b7-f246-4682-9d25-1c0edef0a086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:18.687538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ae2ec8b7-f246-4682-9d25-1c0edef0a086 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:18.688798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ae2ec8b7-f246-4682-9d25-1c0edef0a086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:18.751768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:18.803193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:18.849812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82308864-b301-4b23-93d0-6f5ac4bd30d6 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:18.851677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-82308864-b301-4b23-93d0-6f5ac4bd30d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:19.043329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82308864-b301-4b23-93d0-6f5ac4bd30d6 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:19.130351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:19.175511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9633df-94a5-4e96-8665-f9c00fa3430c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:19.176190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9633df-94a5-4e96-8665-f9c00fa3430c service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:19.176474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9633df-94a5-4e96-8665-f9c00fa3430c service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:19.176748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9633df-94a5-4e96-8665-f9c00fa3430c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:19.178483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4a9633df-94a5-4e96-8665-f9c00fa3430c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:19.279956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9633df-94a5-4e96-8665-f9c00fa3430c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:19.281636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9633df-94a5-4e96-8665-f9c00fa3430c service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:19.303604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 711/1416] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:20.222976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8ee59-666f-488f-84cb-e89c07c71d60 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:20.223616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8ee59-666f-488f-84cb-e89c07c71d60 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:20.223616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8ee59-666f-488f-84cb-e89c07c71d60 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:20.224936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:20.232364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e72da-dddf-49db-afa0-4de2c337d600 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:20.232861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e72da-dddf-49db-afa0-4de2c337d600 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:20.232960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e72da-dddf-49db-afa0-4de2c337d600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:20.328091 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-076e72da-dddf-49db-afa0-4de2c337d600 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:20.329347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-076e72da-dddf-49db-afa0-4de2c337d600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:20.359254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:20.438294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:12:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:20.601540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8ee59-666f-488f-84cb-e89c07c71d60 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:20.603809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4ae8ee59-666f-488f-84cb-e89c07c71d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:20.768047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8ee59-666f-488f-84cb-e89c07c71d60 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:20.815569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:20.897170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7e8c4-d3f5-4326-a3af-bd1b84890e0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:20.897859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7e8c4-d3f5-4326-a3af-bd1b84890e0c service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:20.898149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7e8c4-d3f5-4326-a3af-bd1b84890e0c service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:20.898428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7e8c4-d3f5-4326-a3af-bd1b84890e0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:20.900071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-68b7e8c4-d3f5-4326-a3af-bd1b84890e0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:21.007273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7e8c4-d3f5-4326-a3af-bd1b84890e0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:21.009328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7e8c4-d3f5-4326-a3af-bd1b84890e0c service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:21.029990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 707/1419] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:21.936646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd04bd-5c00-48b8-ad83-b7cc9297dbff None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:21.937426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd04bd-5c00-48b8-ad83-b7cc9297dbff None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:21.937426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd04bd-5c00-48b8-ad83-b7cc9297dbff None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:21.939364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:12:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:21.945414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb1e1c-7386-43b2-925e-fa7ffb542b24 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:21.945934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb1e1c-7386-43b2-925e-fa7ffb542b24 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:21.946031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb1e1c-7386-43b2-925e-fa7ffb542b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:22.006199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:22.034610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-41fb1e1c-7386-43b2-925e-fa7ffb542b24 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:22.035901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-41fb1e1c-7386-43b2-925e-fa7ffb542b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:22.142752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:22.307871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd04bd-5c00-48b8-ad83-b7cc9297dbff None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:22.310133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e7cd04bd-5c00-48b8-ad83-b7cc9297dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:22.498122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd04bd-5c00-48b8-ad83-b7cc9297dbff None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:22.511446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:22.648410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d564f-7a92-43f0-9911-aafbc48c0083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:22.649139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d564f-7a92-43f0-9911-aafbc48c0083 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:22.649426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d564f-7a92-43f0-9911-aafbc48c0083 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:22.649686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d564f-7a92-43f0-9911-aafbc48c0083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:22.651463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cc8d564f-7a92-43f0-9911-aafbc48c0083 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:22.761245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d564f-7a92-43f0-9911-aafbc48c0083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:22.762950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d564f-7a92-43f0-9911-aafbc48c0083 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:22.782280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 714/1422] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:23.662136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08341e66-d5b5-4a73-ae48-48b923851a8b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:23.662715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08341e66-d5b5-4a73-ae48-48b923851a8b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:23.662715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08341e66-d5b5-4a73-ae48-48b923851a8b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:23.664157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:23.670252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ef59-b4b8-4619-b489-0cddad4685a9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:23.670914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ef59-b4b8-4619-b489-0cddad4685a9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:23.670914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ef59-b4b8-4619-b489-0cddad4685a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:23.691404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:23.755097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a491ef59-b4b8-4619-b489-0cddad4685a9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:23.756345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a491ef59-b4b8-4619-b489-0cddad4685a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:23.864829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:24.016210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08341e66-d5b5-4a73-ae48-48b923851a8b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:24.017696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-08341e66-d5b5-4a73-ae48-48b923851a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:24.142506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:24.187360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08341e66-d5b5-4a73-ae48-48b923851a8b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:24.310513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a27f6-6cdd-475b-98a8-197cbad4f31e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:24.311081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a27f6-6cdd-475b-98a8-197cbad4f31e service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:24.311354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a27f6-6cdd-475b-98a8-197cbad4f31e service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:24.311618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a27f6-6cdd-475b-98a8-197cbad4f31e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:24.313305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-547a27f6-6cdd-475b-98a8-197cbad4f31e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:24.396730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:24.411378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a27f6-6cdd-475b-98a8-197cbad4f31e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:24.413026 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a27f6-6cdd-475b-98a8-197cbad4f31e service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:24.433168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 710/1425] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:12:25.338512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37046e9a-e46d-4f60-bbff-4bbf1f49e554 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:25.339217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37046e9a-e46d-4f60-bbff-4bbf1f49e554 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:25.339217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37046e9a-e46d-4f60-bbff-4bbf1f49e554 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:25.340789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:12:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:25.348954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099ac9-b727-4a54-9f4b-a99cbcdda5e7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:25.349417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099ac9-b727-4a54-9f4b-a99cbcdda5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:25.349528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099ac9-b727-4a54-9f4b-a99cbcdda5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:25.440442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-31099ac9-b727-4a54-9f4b-a99cbcdda5e7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:25.442382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-31099ac9-b727-4a54-9f4b-a99cbcdda5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:25.560501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:25.714732 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37046e9a-e46d-4f60-bbff-4bbf1f49e554 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:25.717204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-37046e9a-e46d-4f60-bbff-4bbf1f49e554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:25.821215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:25.891847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37046e9a-e46d-4f60-bbff-4bbf1f49e554 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:26.020888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685f13b-9673-4cd3-a31b-4a2433475d6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:26.021550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685f13b-9673-4cd3-a31b-4a2433475d6e service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:26.021865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685f13b-9673-4cd3-a31b-4a2433475d6e service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:26.022139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685f13b-9673-4cd3-a31b-4a2433475d6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:26.023818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b685f13b-9673-4cd3-a31b-4a2433475d6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:26.093284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:26.140597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685f13b-9673-4cd3-a31b-4a2433475d6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:26.142335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b685f13b-9673-4cd3-a31b-4a2433475d6e service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:26.164884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 717/1428] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:27.006011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c88fb4-bdb8-4a60-8bc3-ee526cdf8a1c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:27.006625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c88fb4-bdb8-4a60-8bc3-ee526cdf8a1c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:27.006625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c88fb4-bdb8-4a60-8bc3-ee526cdf8a1c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:27.008040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:27.014520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cf38f-c8e2-47ae-9d84-fb6ce60d6c2f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:27.015088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cf38f-c8e2-47ae-9d84-fb6ce60d6c2f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:27.015088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cf38f-c8e2-47ae-9d84-fb6ce60d6c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:27.098012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e93cf38f-c8e2-47ae-9d84-fb6ce60d6c2f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:27.099268 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e93cf38f-c8e2-47ae-9d84-fb6ce60d6c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:27.215042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:27.370858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c88fb4-bdb8-4a60-8bc3-ee526cdf8a1c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:27.372854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-22c88fb4-bdb8-4a60-8bc3-ee526cdf8a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:27.434927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:27.541087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c88fb4-bdb8-4a60-8bc3-ee526cdf8a1c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:27.676499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a99ac5-053d-4a79-bd57-86830ec7855b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:27.677165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a99ac5-053d-4a79-bd57-86830ec7855b service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:27.677452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a99ac5-053d-4a79-bd57-86830ec7855b service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:27.677773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a99ac5-053d-4a79-bd57-86830ec7855b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:27.679440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-73a99ac5-053d-4a79-bd57-86830ec7855b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:27.688168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:27.789295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a99ac5-053d-4a79-bd57-86830ec7855b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:27.790974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a99ac5-053d-4a79-bd57-86830ec7855b service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:27.810686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 713/1431] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:28.643355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfb78e-296e-4f12-ae66-25a9f8452a2e None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:28.643843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfb78e-296e-4f12-ae66-25a9f8452a2e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:28.643953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfb78e-296e-4f12-ae66-25a9f8452a2e None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:28.645300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:28.651623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dd271-7834-4bd4-9f2b-7d15c8f98aa6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:28.652116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dd271-7834-4bd4-9f2b-7d15c8f98aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:28.652218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dd271-7834-4bd4-9f2b-7d15c8f98aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:28.748409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6e2dd271-7834-4bd4-9f2b-7d15c8f98aa6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:28.749690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6e2dd271-7834-4bd4-9f2b-7d15c8f98aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:28.839970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:28.881989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:29.040764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfb78e-296e-4f12-ae66-25a9f8452a2e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:29.042571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5ddfb78e-296e-4f12-ae66-25a9f8452a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:29.062271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:29.206991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfb78e-296e-4f12-ae66-25a9f8452a2e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:29.315752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:29.346033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813218ce-d85d-4c5d-9201-541ab7b4e6c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:29.347083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813218ce-d85d-4c5d-9201-541ab7b4e6c9 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:29.347551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813218ce-d85d-4c5d-9201-541ab7b4e6c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:29.347841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813218ce-d85d-4c5d-9201-541ab7b4e6c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:29.350353 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-813218ce-d85d-4c5d-9201-541ab7b4e6c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:29.464849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813218ce-d85d-4c5d-9201-541ab7b4e6c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:29.466487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813218ce-d85d-4c5d-9201-541ab7b4e6c9 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:29.486356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 720/1434] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:30.309917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9f979-fc13-490a-8720-0e14d47efc8a None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.310586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9f979-fc13-490a-8720-0e14d47efc8a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.310814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9f979-fc13-490a-8720-0e14d47efc8a None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.312264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:30.318021 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493cb1e-beb3-4075-8849-e410c792c5e2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:30.318587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493cb1e-beb3-4075-8849-e410c792c5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:30.318587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493cb1e-beb3-4075-8849-e410c792c5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:30.408582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4493cb1e-beb3-4075-8849-e410c792c5e2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:30.409942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4493cb1e-beb3-4075-8849-e410c792c5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:30.462044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:30.550889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:30.707110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9f979-fc13-490a-8720-0e14d47efc8a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:30.708871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-70f9f979-fc13-490a-8720-0e14d47efc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:30.876537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9f979-fc13-490a-8720-0e14d47efc8a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:30.934892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:31.006702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121ee8f2-ec71-4095-8ca9-876331030d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:31.007379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ee8f2-ec71-4095-8ca9-876331030d41 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:31.007665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ee8f2-ec71-4095-8ca9-876331030d41 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:31.007934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121ee8f2-ec71-4095-8ca9-876331030d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:31.009616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-121ee8f2-ec71-4095-8ca9-876331030d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:31.112468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ee8f2-ec71-4095-8ca9-876331030d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:31.114159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121ee8f2-ec71-4095-8ca9-876331030d41 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:31.133001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 716/1437] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:31.973167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58b382-1713-41dc-855a-9b2beb3fb8eb None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:31.973746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58b382-1713-41dc-855a-9b2beb3fb8eb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:31.973746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58b382-1713-41dc-855a-9b2beb3fb8eb None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:31.975029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:31.980644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a219a-7d63-43b6-9c06-57856f19a726 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:31.981482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a219a-7d63-43b6-9c06-57856f19a726 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:31.981482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a219a-7d63-43b6-9c06-57856f19a726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:32.074998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-860a219a-7d63-43b6-9c06-57856f19a726 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:32.076412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-860a219a-7d63-43b6-9c06-57856f19a726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:32.078808 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:32.184189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:12:32.334211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58b382-1713-41dc-855a-9b2beb3fb8eb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:32.335710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7f58b382-1713-41dc-855a-9b2beb3fb8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:32.502902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58b382-1713-41dc-855a-9b2beb3fb8eb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:32.534564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:32.629747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f6b0c-84eb-427f-b830-fecba43fbdec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:32.630433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f6b0c-84eb-427f-b830-fecba43fbdec service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:32.630714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f6b0c-84eb-427f-b830-fecba43fbdec service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:32.630991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f6b0c-84eb-427f-b830-fecba43fbdec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:32.632680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bf7f6b0c-84eb-427f-b830-fecba43fbdec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:32.732976 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f6b0c-84eb-427f-b830-fecba43fbdec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:32.734679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f6b0c-84eb-427f-b830-fecba43fbdec service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:32.755190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 723/1440] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:34.772000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0229f-515b-4b29-9bef-e68a54b11897 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:34.772625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0229f-515b-4b29-9bef-e68a54b11897 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:34.772625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0229f-515b-4b29-9bef-e68a54b11897 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:34.774540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:34.780478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a56e9f-9f6a-41d3-9ab1-5421813765cb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:34.781013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a56e9f-9f6a-41d3-9ab1-5421813765cb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:34.781352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a56e9f-9f6a-41d3-9ab1-5421813765cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:34.831761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:34.862391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-97a56e9f-9f6a-41d3-9ab1-5421813765cb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:34.863644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-97a56e9f-9f6a-41d3-9ab1-5421813765cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:34.978883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:35.023925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0229f-515b-4b29-9bef-e68a54b11897 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:35.025911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a2b0229f-515b-4b29-9bef-e68a54b11897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:35.145863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0229f-515b-4b29-9bef-e68a54b11897 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:35.182817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:35.228459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a23d9-f5b9-434b-9380-db17eb4ce311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:35.229244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a23d9-f5b9-434b-9380-db17eb4ce311 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:35.229489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a23d9-f5b9-434b-9380-db17eb4ce311 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:35.229838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a23d9-f5b9-434b-9380-db17eb4ce311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:35.231600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9c6a23d9-f5b9-434b-9380-db17eb4ce311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:35.348017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a23d9-f5b9-434b-9380-db17eb4ce311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:35.349477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a23d9-f5b9-434b-9380-db17eb4ce311 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:35.368727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 719/1443] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:37.665244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9f243-e4ea-46a3-ab43-d5bc76743a1c None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:37.665821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9f243-e4ea-46a3-ab43-d5bc76743a1c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:37.665821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9f243-e4ea-46a3-ab43-d5bc76743a1c None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:37.667431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:37.673429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743dedf-25da-453e-b556-27c0a97bed9f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:37.673966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743dedf-25da-453e-b556-27c0a97bed9f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:37.674090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743dedf-25da-453e-b556-27c0a97bed9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:37.744484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1743dedf-25da-453e-b556-27c0a97bed9f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:37.745803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1743dedf-25da-453e-b556-27c0a97bed9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:37.759194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:37.850086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:37.889553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9f243-e4ea-46a3-ab43-d5bc76743a1c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:37.891104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d7e9f243-e4ea-46a3-ab43-d5bc76743a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:38.004555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9f243-e4ea-46a3-ab43-d5bc76743a1c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:38.017360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:38.087833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d62b5c4-ae3e-4d54-989e-e4a8448d925a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:38.088834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62b5c4-ae3e-4d54-989e-e4a8448d925a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:38.089551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62b5c4-ae3e-4d54-989e-e4a8448d925a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:38.089865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d62b5c4-ae3e-4d54-989e-e4a8448d925a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:38.091629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6d62b5c4-ae3e-4d54-989e-e4a8448d925a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:38.220821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62b5c4-ae3e-4d54-989e-e4a8448d925a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:38.222316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d62b5c4-ae3e-4d54-989e-e4a8448d925a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:38.242049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 726/1446] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:40.831932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54019c-c5b9-4124-bebe-bb62490c29e9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:40.832558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54019c-c5b9-4124-bebe-bb62490c29e9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:40.832558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca54019c-c5b9-4124-bebe-bb62490c29e9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:40.833938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:40.840138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe629-4efe-4126-84e0-f4e55800c0b1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:40.840708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe629-4efe-4126-84e0-f4e55800c0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:40.840708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ffe629-4efe-4126-84e0-f4e55800c0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:40.900062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:40.922680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-03ffe629-4efe-4126-84e0-f4e55800c0b1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:40.923932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-03ffe629-4efe-4126-84e0-f4e55800c0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:41.040238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:41.080357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54019c-c5b9-4124-bebe-bb62490c29e9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:41.082132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ca54019c-c5b9-4124-bebe-bb62490c29e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:41.201432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca54019c-c5b9-4124-bebe-bb62490c29e9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:41.247163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:41.297217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87037db-e42f-482d-b4c2-d2fa2c191894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:41.298068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87037db-e42f-482d-b4c2-d2fa2c191894 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:41.298377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87037db-e42f-482d-b4c2-d2fa2c191894 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:41.298764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87037db-e42f-482d-b4c2-d2fa2c191894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:41.300789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f87037db-e42f-482d-b4c2-d2fa2c191894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:41.423304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87037db-e42f-482d-b4c2-d2fa2c191894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:41.424767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87037db-e42f-482d-b4c2-d2fa2c191894 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:41.447416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 722/1449] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:43.649874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc67d52-a354-409a-b8af-fdbf02b822cc None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:43.650485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc67d52-a354-409a-b8af-fdbf02b822cc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:43.650595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc67d52-a354-409a-b8af-fdbf02b822cc None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:43.652588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:43.659710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34c10-10f2-4a59-83b9-f8fb51571056 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:43.660246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34c10-10f2-4a59-83b9-f8fb51571056 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:43.660690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e34c10-10f2-4a59-83b9-f8fb51571056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:43.749056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-67e34c10-10f2-4a59-83b9-f8fb51571056 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:43.750236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-67e34c10-10f2-4a59-83b9-f8fb51571056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:43.778067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:43.875940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:43.921186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc67d52-a354-409a-b8af-fdbf02b822cc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:43.922848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cfc67d52-a354-409a-b8af-fdbf02b822cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:44.045296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc67d52-a354-409a-b8af-fdbf02b822cc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:44.069546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:44.131169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01602ed0-dc90-44bf-aecd-df2cc1d51b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:44.131862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01602ed0-dc90-44bf-aecd-df2cc1d51b27 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:44.132150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01602ed0-dc90-44bf-aecd-df2cc1d51b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:44.132433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01602ed0-dc90-44bf-aecd-df2cc1d51b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:44.134206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-01602ed0-dc90-44bf-aecd-df2cc1d51b27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:44.264110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01602ed0-dc90-44bf-aecd-df2cc1d51b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:44.265617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01602ed0-dc90-44bf-aecd-df2cc1d51b27 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:44.287930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 729/1452] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:46.492809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a73c3f-e612-4f39-a1e4-4da0a9e4e09b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:46.493380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a73c3f-e612-4f39-a1e4-4da0a9e4e09b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:46.493380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a73c3f-e612-4f39-a1e4-4da0a9e4e09b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:46.495278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:46.502009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf10d0d-dc7b-4264-b4ab-98591ce9ba13 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:46.502798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf10d0d-dc7b-4264-b4ab-98591ce9ba13 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:46.502798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf10d0d-dc7b-4264-b4ab-98591ce9ba13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:46.571539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:46.575546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ebf10d0d-dc7b-4264-b4ab-98591ce9ba13 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:46.577053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ebf10d0d-dc7b-4264-b4ab-98591ce9ba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:46.686763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:46.726259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a73c3f-e612-4f39-a1e4-4da0a9e4e09b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:46.727892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-66a73c3f-e612-4f39-a1e4-4da0a9e4e09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:46.850497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a73c3f-e612-4f39-a1e4-4da0a9e4e09b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:46.906358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:46.933178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b815c3-b207-437c-aca1-4d5c7b24de82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:46.933885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b815c3-b207-437c-aca1-4d5c7b24de82 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:46.934174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b815c3-b207-437c-aca1-4d5c7b24de82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:46.934455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b815c3-b207-437c-aca1-4d5c7b24de82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:46.936155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-33b815c3-b207-437c-aca1-4d5c7b24de82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:47.054579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b815c3-b207-437c-aca1-4d5c7b24de82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:47.056053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b815c3-b207-437c-aca1-4d5c7b24de82 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:47.075191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 725/1455] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:49.239196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2f5e5-8605-469b-8266-ff42cd8786b9 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:49.239844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2f5e5-8605-469b-8266-ff42cd8786b9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:49.239844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2f5e5-8605-469b-8266-ff42cd8786b9 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:49.241157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:12:49.253379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4fd5-7890-4373-a0c2-2c5614443f5c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:49.254152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4fd5-7890-4373-a0c2-2c5614443f5c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:49.254152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4fd5-7890-4373-a0c2-2c5614443f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:49.337507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-aabc4fd5-7890-4373-a0c2-2c5614443f5c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:49.339033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-aabc4fd5-7890-4373-a0c2-2c5614443f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:49.371382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:49.444987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:49.486047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2f5e5-8605-469b-8266-ff42cd8786b9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:49.488573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d7d2f5e5-8605-469b-8266-ff42cd8786b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:49.613059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2f5e5-8605-469b-8266-ff42cd8786b9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:49.659386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:49.716851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c930438a-2009-4433-8939-48a945810e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:49.717522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930438a-2009-4433-8939-48a945810e99 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:49.717838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930438a-2009-4433-8939-48a945810e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:49.718112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c930438a-2009-4433-8939-48a945810e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:49.719839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c930438a-2009-4433-8939-48a945810e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:49.870421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930438a-2009-4433-8939-48a945810e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:49.871922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c930438a-2009-4433-8939-48a945810e99 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:49.894264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 732/1458] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:52.043099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167742af-0c57-4d56-80db-6c598a29917c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:52.043648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167742af-0c57-4d56-80db-6c598a29917c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:52.043648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167742af-0c57-4d56-80db-6c598a29917c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:52.044980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:52.050913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de590-ff96-47ec-ba37-df22c6638656 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:52.051439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de590-ff96-47ec-ba37-df22c6638656 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:52.051439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4de590-ff96-47ec-ba37-df22c6638656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:52.126965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1d4de590-ff96-47ec-ba37-df22c6638656 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:52.128223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1d4de590-ff96-47ec-ba37-df22c6638656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:52.137901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:52.255195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:52.293567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167742af-0c57-4d56-80db-6c598a29917c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:52.295078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-167742af-0c57-4d56-80db-6c598a29917c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:52.407331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167742af-0c57-4d56-80db-6c598a29917c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:52.468646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:52.498532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89d089c-90eb-47f7-a4ef-b39620ca816f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:52.499197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d089c-90eb-47f7-a4ef-b39620ca816f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:52.499482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d089c-90eb-47f7-a4ef-b39620ca816f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:52.499736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d089c-90eb-47f7-a4ef-b39620ca816f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:52.501439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c89d089c-90eb-47f7-a4ef-b39620ca816f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:52.635024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d089c-90eb-47f7-a4ef-b39620ca816f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:52.637022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89d089c-90eb-47f7-a4ef-b39620ca816f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:52.656680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 728/1461] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:54.847928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209e40-8414-4f4e-90ad-e91834163451 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:54.848526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209e40-8414-4f4e-90ad-e91834163451 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:54.848526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e209e40-8414-4f4e-90ad-e91834163451 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:54.849918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:54.855991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02679692-7b62-4a4b-a557-43f5a9b6eabb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:54.856498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02679692-7b62-4a4b-a557-43f5a9b6eabb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:54.856608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02679692-7b62-4a4b-a557-43f5a9b6eabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:54.929307 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-02679692-7b62-4a4b-a557-43f5a9b6eabb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:54.930483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-02679692-7b62-4a4b-a557-43f5a9b6eabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:54.986525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:55.056409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:12:55.095954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e209e40-8414-4f4e-90ad-e91834163451 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:55.097685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7e209e40-8414-4f4e-90ad-e91834163451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:55.242130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e209e40-8414-4f4e-90ad-e91834163451 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:55.321733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:55.390258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f55696-58dc-4b1b-93ec-b74dd0547e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:55.391395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55696-58dc-4b1b-93ec-b74dd0547e1b service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:55.391900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55696-58dc-4b1b-93ec-b74dd0547e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:55.392404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f55696-58dc-4b1b-93ec-b74dd0547e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:55.395727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-10f55696-58dc-4b1b-93ec-b74dd0547e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:55.602912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f55696-58dc-4b1b-93ec-b74dd0547e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:55.605559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f55696-58dc-4b1b-93ec-b74dd0547e1b service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:55.639265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 735/1464] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:12:57.769995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a921fdf-1b84-4036-8033-5f888e3f63e3 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:57.770724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a921fdf-1b84-4036-8033-5f888e3f63e3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:57.770724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a921fdf-1b84-4036-8033-5f888e3f63e3 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:57.771920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:12:57.780150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dddf3-b515-45c6-b254-84219d034807 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:57.780791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dddf3-b515-45c6-b254-84219d034807 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:57.781023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dddf3-b515-45c6-b254-84219d034807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:57.876075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-821dddf3-b515-45c6-b254-84219d034807 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:12:57.877343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-821dddf3-b515-45c6-b254-84219d034807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:57.899785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:58.013238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:12:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:12:58.052760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a921fdf-1b84-4036-8033-5f888e3f63e3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:12:58.054394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0a921fdf-1b84-4036-8033-5f888e3f63e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:58.193499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a921fdf-1b84-4036-8033-5f888e3f63e3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:12:58.281786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:12:58.289850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ec1a3-7188-49ab-88f3-b59634fc49ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:12:58.290511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec1a3-7188-49ab-88f3-b59634fc49ed service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:12:58.290791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec1a3-7188-49ab-88f3-b59634fc49ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:12:58.291059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ec1a3-7188-49ab-88f3-b59634fc49ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:12:58.292841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-265ec1a3-7188-49ab-88f3-b59634fc49ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:12:58.424484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ec1a3-7188-49ab-88f3-b59634fc49ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:12:58.425897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ec1a3-7188-49ab-88f3-b59634fc49ed service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:12:58.447097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 731/1467] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:00.577580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ea00-402d-4066-877c-443659fbfa98 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:00.578337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ea00-402d-4066-877c-443659fbfa98 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:00.578337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029ea00-402d-4066-877c-443659fbfa98 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:00.579744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:00.586333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee89391-9d40-4904-9f82-44272f13dfd8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:00.587125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee89391-9d40-4904-9f82-44272f13dfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:00.587125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee89391-9d40-4904-9f82-44272f13dfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:00.688218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1ee89391-9d40-4904-9f82-44272f13dfd8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:00.690471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1ee89391-9d40-4904-9f82-44272f13dfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:00.782406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:00.843602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:13:00.882817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029ea00-402d-4066-877c-443659fbfa98 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:00.884446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a029ea00-402d-4066-877c-443659fbfa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:01.067039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029ea00-402d-4066-877c-443659fbfa98 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:01.159727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:01.193557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be0a26-5ff6-4da8-ac2d-31c2f1a5f1ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:01.194281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be0a26-5ff6-4da8-ac2d-31c2f1a5f1ec service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:01.194558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be0a26-5ff6-4da8-ac2d-31c2f1a5f1ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:01.194815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be0a26-5ff6-4da8-ac2d-31c2f1a5f1ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:01.196623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a2be0a26-5ff6-4da8-ac2d-31c2f1a5f1ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:01.342744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be0a26-5ff6-4da8-ac2d-31c2f1a5f1ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:01.344241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be0a26-5ff6-4da8-ac2d-31c2f1a5f1ec service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:01.366722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 738/1470] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:13:10.841250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=22 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:11.363941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=66 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:21.927477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed4ad7-c47a-4872-bb91-6012b050067c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:21.928190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed4ad7-c47a-4872-bb91-6012b050067c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:21.928190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed4ad7-c47a-4872-bb91-6012b050067c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:21.929340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 733/1471] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:13:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:21.935654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250f0f0-7bc9-416a-a5f4-6f8209838350 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:21.936145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250f0f0-7bc9-416a-a5f4-6f8209838350 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:21.936251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250f0f0-7bc9-416a-a5f4-6f8209838350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:22.006324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d250f0f0-7bc9-416a-a5f4-6f8209838350 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:22.009738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d250f0f0-7bc9-416a-a5f4-6f8209838350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:22.114697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 739/1472] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:13:22.178230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce481fa-e882-40b2-82a2-53c96f3f7d1f None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:22.178476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce481fa-e882-40b2-82a2-53c96f3f7d1f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:22.178784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce481fa-e882-40b2-82a2-53c96f3f7d1f None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:22.180140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 734/1473] 10.222.0.71 () {54 vars in 961 bytes} [Mon Jul 18 14:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:22.185906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0335e-4f75-4071-ac93-0da8de732495 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:22.186207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0335e-4f75-4071-ac93-0da8de732495 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:22.186484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b0335e-4f75-4071-ac93-0da8de732495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:22.266711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d6b0335e-4f75-4071-ac93-0da8de732495 None None] MFA Rules not processed for user `d661615b2dd94d9db6183f580ab0fee6`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:22.267981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d6b0335e-4f75-4071-ac93-0da8de732495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:22.290976 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:22.349193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 740/1474] 10.222.0.71 () {58 vars in 1058 bytes} [Mon Jul 18 14:13:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:13:22.357345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce481fa-e882-40b2-82a2-53c96f3f7d1f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:22.358909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cce481fa-e882-40b2-82a2-53c96f3f7d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:22.470592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce481fa-e882-40b2-82a2-53c96f3f7d1f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:22.550728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680157f-caee-4593-94bf-0fa233ab5b47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:22.551407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680157f-caee-4593-94bf-0fa233ab5b47 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:22.551686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680157f-caee-4593-94bf-0fa233ab5b47 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:22.551966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680157f-caee-4593-94bf-0fa233ab5b47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:22.553680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9680157f-caee-4593-94bf-0fa233ab5b47 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:22.616002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:22.686402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680157f-caee-4593-94bf-0fa233ab5b47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:22.687918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9680157f-caee-4593-94bf-0fa233ab5b47 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:22.709031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 735/1475] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:24.967482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b80b7b-83e7-472c-88c9-d4d877830be5 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:24.967985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b80b7b-83e7-472c-88c9-d4d877830be5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:24.968102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b80b7b-83e7-472c-88c9-d4d877830be5 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:24.969468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 741/1476] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:24.974773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eff6d3-b397-47c3-98ca-ea858834db92 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:24.975228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eff6d3-b397-47c3-98ca-ea858834db92 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:24.975379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eff6d3-b397-47c3-98ca-ea858834db92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:25.043264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-60eff6d3-b397-47c3-98ca-ea858834db92 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:25.044500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-60eff6d3-b397-47c3-98ca-ea858834db92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:25.145742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 736/1477] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:13:25.169279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b80b7b-83e7-472c-88c9-d4d877830be5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:25.170904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d6b80b7b-83e7-472c-88c9-d4d877830be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:25.325729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b80b7b-83e7-472c-88c9-d4d877830be5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:25.378659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:25.432671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae62c95-c574-40cf-9328-4d255658a294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:25.433374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae62c95-c574-40cf-9328-4d255658a294 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:25.433696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae62c95-c574-40cf-9328-4d255658a294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:25.433972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae62c95-c574-40cf-9328-4d255658a294 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:25.434695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae62c95-c574-40cf-9328-4d255658a294 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:25.436626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-5ae62c95-c574-40cf-9328-4d255658a294 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:13:25.436626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jul 18 14:13:25.436626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: Jul 18 14:13:25.437858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae62c95-c574-40cf-9328-4d255658a294 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:25.459239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 742/1478] 10.222.0.71 () {56 vars in 1108 bytes} [Mon Jul 18 14:13:25 2022] GET /identity/v3/projects => generated 2553 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:13:27.578476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b30ad2-2daa-445d-81eb-11c760e7d0a9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:27.579670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b30ad2-2daa-445d-81eb-11c760e7d0a9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:27.579670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b30ad2-2daa-445d-81eb-11c760e7d0a9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:27.581776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 737/1479] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:13:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:27.590393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec47797-bb66-4ff0-8c1f-c3152abc5b4b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:27.591030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec47797-bb66-4ff0-8c1f-c3152abc5b4b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:27.591030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec47797-bb66-4ff0-8c1f-c3152abc5b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:27.666763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8ec47797-bb66-4ff0-8c1f-c3152abc5b4b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:27.668412 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8ec47797-bb66-4ff0-8c1f-c3152abc5b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:27.786302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 743/1480] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:13:27.832039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b30ad2-2daa-445d-81eb-11c760e7d0a9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:27.833626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-95b30ad2-2daa-445d-81eb-11c760e7d0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:27.845940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:27.995690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b30ad2-2daa-445d-81eb-11c760e7d0a9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:28.087861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:28.127695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b9b06-d4a9-4902-8de3-0df1e238b3a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:28.128352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b9b06-d4a9-4902-8de3-0df1e238b3a8 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:28.128634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b9b06-d4a9-4902-8de3-0df1e238b3a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:28.128906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b9b06-d4a9-4902-8de3-0df1e238b3a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:28.130590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c1b9b06-d4a9-4902-8de3-0df1e238b3a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:28.239020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b9b06-d4a9-4902-8de3-0df1e238b3a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:28.241416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b9b06-d4a9-4902-8de3-0df1e238b3a8 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:28.262302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 738/1481] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:30.470184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22568e74-0f4d-45ec-a8e2-cba94df7b3d4 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:30.470788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22568e74-0f4d-45ec-a8e2-cba94df7b3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:30.470788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22568e74-0f4d-45ec-a8e2-cba94df7b3d4 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:30.472198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 744/1482] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:30.480245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a35844-8e33-4b4f-a4f2-b75d0b406964 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:30.480759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a35844-8e33-4b4f-a4f2-b75d0b406964 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:30.480759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a35844-8e33-4b4f-a4f2-b75d0b406964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:30.686457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-87a35844-8e33-4b4f-a4f2-b75d0b406964 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:30.687750 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-87a35844-8e33-4b4f-a4f2-b75d0b406964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:30.739650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:30.794405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 739/1483] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:13:30.834214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22568e74-0f4d-45ec-a8e2-cba94df7b3d4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:30.836371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-22568e74-0f4d-45ec-a8e2-cba94df7b3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:30.869384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:30.960559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22568e74-0f4d-45ec-a8e2-cba94df7b3d4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:31.044894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d531997-4c3e-451a-b6fc-0b2696ae79ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:31.045451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531997-4c3e-451a-b6fc-0b2696ae79ae service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:31.045752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531997-4c3e-451a-b6fc-0b2696ae79ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:31.046028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531997-4c3e-451a-b6fc-0b2696ae79ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:31.047685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5d531997-4c3e-451a-b6fc-0b2696ae79ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:31.136838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:31.178087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d531997-4c3e-451a-b6fc-0b2696ae79ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:31.179565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d531997-4c3e-451a-b6fc-0b2696ae79ae service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:31.199302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 745/1484] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:33.329112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5859f-d767-4155-bbcb-25c6ee75325b None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:33.329776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5859f-d767-4155-bbcb-25c6ee75325b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:33.329776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5859f-d767-4155-bbcb-25c6ee75325b None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:33.331153 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 740/1485] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:33.337355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a8e73-be33-4f49-a7c4-cf2b2fda8c04 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:33.337987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a8e73-be33-4f49-a7c4-cf2b2fda8c04 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:33.337987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a8e73-be33-4f49-a7c4-cf2b2fda8c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:33.422386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7f7a8e73-be33-4f49-a7c4-cf2b2fda8c04 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:33.423607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7f7a8e73-be33-4f49-a7c4-cf2b2fda8c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:33.530379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 746/1486] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:13:33.570280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5859f-d767-4155-bbcb-25c6ee75325b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:33.572702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3fe5859f-d767-4155-bbcb-25c6ee75325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:33.682829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5859f-d767-4155-bbcb-25c6ee75325b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:33.772800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:33.775442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9250e3-7089-4fbb-ba17-9a1a50bc8968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:33.776495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9250e3-7089-4fbb-ba17-9a1a50bc8968 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:33.776969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9250e3-7089-4fbb-ba17-9a1a50bc8968 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:33.777708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9250e3-7089-4fbb-ba17-9a1a50bc8968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:33.780544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-eb9250e3-7089-4fbb-ba17-9a1a50bc8968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:33.906519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9250e3-7089-4fbb-ba17-9a1a50bc8968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:33.907918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9250e3-7089-4fbb-ba17-9a1a50bc8968 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:33.928054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 741/1487] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:36.225761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9d29d-51c3-4a22-bf53-70c0ad873194 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:36.226300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9d29d-51c3-4a22-bf53-70c0ad873194 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:36.226300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9d29d-51c3-4a22-bf53-70c0ad873194 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:36.227661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 747/1488] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:36.233562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1494-08bd-4255-b210-36cf8443c19b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:36.234196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1494-08bd-4255-b210-36cf8443c19b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:36.234196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3b1494-08bd-4255-b210-36cf8443c19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:36.304350 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-df3b1494-08bd-4255-b210-36cf8443c19b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:36.305629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-df3b1494-08bd-4255-b210-36cf8443c19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:36.371204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:36.419521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 742/1489] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:13:36.460435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9d29d-51c3-4a22-bf53-70c0ad873194 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:36.462048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-75a9d29d-51c3-4a22-bf53-70c0ad873194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:36.502251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:36.574946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9d29d-51c3-4a22-bf53-70c0ad873194 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:36.656302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ad5404-bbcd-4867-ac83-043231ede77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:36.656970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad5404-bbcd-4867-ac83-043231ede77a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:36.657250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad5404-bbcd-4867-ac83-043231ede77a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:36.657524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad5404-bbcd-4867-ac83-043231ede77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:36.659370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-77ad5404-bbcd-4867-ac83-043231ede77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:36.756830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:36.790605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad5404-bbcd-4867-ac83-043231ede77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:36.792114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ad5404-bbcd-4867-ac83-043231ede77a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:36.812719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 748/1490] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:39.107327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da943e1-9a89-4cc8-9e46-b689829f10e1 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:39.107890 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da943e1-9a89-4cc8-9e46-b689829f10e1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:39.107890 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da943e1-9a89-4cc8-9e46-b689829f10e1 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:39.109171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 743/1491] 10.222.0.71 () {54 vars in 926 bytes} [Mon Jul 18 14:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:13:39.115314 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65fde1-4b21-4cc2-a895-b52fb277f011 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:39.115777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65fde1-4b21-4cc2-a895-b52fb277f011 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:39.115891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65fde1-4b21-4cc2-a895-b52fb277f011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:39.188024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8e65fde1-4b21-4cc2-a895-b52fb277f011 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:13:39.189288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8e65fde1-4b21-4cc2-a895-b52fb277f011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:39.316019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 749/1492] 10.222.0.71 () {58 vars in 1023 bytes} [Mon Jul 18 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:13:39.356020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da943e1-9a89-4cc8-9e46-b689829f10e1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:13:39.357794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2da943e1-9a89-4cc8-9e46-b689829f10e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:39.468118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da943e1-9a89-4cc8-9e46-b689829f10e1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:13:39.547019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b986f70d-459b-4c92-802d-10104a467bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:13:39.547690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986f70d-459b-4c92-802d-10104a467bff service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:13:39.547983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986f70d-459b-4c92-802d-10104a467bff service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:13:39.548257 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b986f70d-459b-4c92-802d-10104a467bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:13:39.550698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b986f70d-459b-4c92-802d-10104a467bff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:13:39.561789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:39.686693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986f70d-459b-4c92-802d-10104a467bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:13:39.688134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b986f70d-459b-4c92-802d-10104a467bff service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:13:39.706622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 744/1493] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:13:49.314084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=82 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:13:49.704217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=50 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:10.852531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee007a-117e-4d78-850a-5e5fa48671fb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:10.853166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee007a-117e-4d78-850a-5e5fa48671fb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:10.853166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee007a-117e-4d78-850a-5e5fa48671fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:10.968368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7eee007a-117e-4d78-850a-5e5fa48671fb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:10.970727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7eee007a-117e-4d78-850a-5e5fa48671fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:11.107308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 750/1494] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:15:11.116177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afe50-b80c-494c-9107-03ea3e4d6e65 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:11.116838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afe50-b80c-494c-9107-03ea3e4d6e65 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:11.116838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613afe50-b80c-494c-9107-03ea3e4d6e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:11.210001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-613afe50-b80c-494c-9107-03ea3e4d6e65 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:11.211297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-613afe50-b80c-494c-9107-03ea3e4d6e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:11.331818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 745/1495] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:15:11.336752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee007a-117e-4d78-850a-5e5fa48671fb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:11.338035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7eee007a-117e-4d78-850a-5e5fa48671fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:11.350710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee007a-117e-4d78-850a-5e5fa48671fb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:11.421040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:11.459954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:11.460620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:11.460949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:11.461205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:11.462023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:15:11.462360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:11.463417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:15:11.463417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jul 18 14:15:11.463417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: Jul 18 14:15:11.464576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:11.477858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 751/1496] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:15:11.487764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613afe50-b80c-494c-9107-03ea3e4d6e65 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:11.502354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613afe50-b80c-494c-9107-03ea3e4d6e65 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:11.583502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:11.625860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:11.626554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:11.626826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:11.627095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:11.627806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:11.629707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:15:11.629707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jul 18 14:15:11.629707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: Jul 18 14:15:11.630922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:11.700978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 746/1497] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:15:11 2022] POST /identity/v3/projects => generated 379 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:15:11.705505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:11.707133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:11.753482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9426af-f1bb-40d8-989b-c518faa34662 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:11.857857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:11.858797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:11.858797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:11.859112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:11.859763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:11.860719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:15:11.860719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jul 18 14:15:11.860719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: Jul 18 14:15:11.861941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:11.910258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 752/1498] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:15:11 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:15:11.915381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:11.917115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:11.989954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449f3b1-e835-4754-892e-b94d63fad4be admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:12.097205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:12.113550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:12.114201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:12.114908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:12.115325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:12.116124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:12.117457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:15:12.117457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jul 18 14:15:12.117457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: Jul 18 14:15:12.118478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:12.135021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 747/1499] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:15:12.139848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:12.141531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:12.154350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22be83e-7b47-4a1c-bfd7-a7af0ec2c9d7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:12.184590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:12.268869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e196e6-f65f-4400-b6aa-7a70b7133413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:12.269571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e196e6-f65f-4400-b6aa-7a70b7133413 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:12.269882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e196e6-f65f-4400-b6aa-7a70b7133413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:12.270155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e196e6-f65f-4400-b6aa-7a70b7133413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:12.270888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e196e6-f65f-4400-b6aa-7a70b7133413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:12.271913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-b8e196e6-f65f-4400-b6aa-7a70b7133413 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:15:12.271913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jul 18 14:15:12.271913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: Jul 18 14:15:12.272801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e196e6-f65f-4400-b6aa-7a70b7133413 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:12.286381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 753/1500] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:15:12.290882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:12.292480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:12.329956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25b15c-b778-4a3b-acf8-103d47078ab6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:12.434475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d38fc5-3526-439c-86cb-ef9062c28830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:12.435385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d38fc5-3526-439c-86cb-ef9062c28830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:12.435706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d38fc5-3526-439c-86cb-ef9062c28830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:12.435964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d38fc5-3526-439c-86cb-ef9062c28830 admin admin] PATH_INFO: `/v3/projects/9be8edd85a844dcd895c5a04d229850d/users/963c3fb0b8a64a3dbac7a36c55d16d04/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:12.460189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d38fc5-3526-439c-86cb-ef9062c28830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be8edd85a844dcd895c5a04d229850d, user_id=963c3fb0b8a64a3dbac7a36c55d16d04, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:12.461268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-a3d38fc5-3526-439c-86cb-ef9062c28830 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:15:12.461268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jul 18 14:15:12.461268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: Jul 18 14:15:12.463374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d38fc5-3526-439c-86cb-ef9062c28830 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:12.495719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 748/1501] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:15:12 2022] PUT /identity/v3/projects/9be8edd85a844dcd895c5a04d229850d/users/963c3fb0b8a64a3dbac7a36c55d16d04/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:15:12.509579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed8968-e63f-4c24-b5f4-1b5e59bfe8c2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:12.509907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed8968-e63f-4c24-b5f4-1b5e59bfe8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:12.510395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed8968-e63f-4c24-b5f4-1b5e59bfe8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:12.573792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2eed8968-e63f-4c24-b5f4-1b5e59bfe8c2 None None] MFA Rules not processed for user `963c3fb0b8a64a3dbac7a36c55d16d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:12.575107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2eed8968-e63f-4c24-b5f4-1b5e59bfe8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:12.664723 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 754/1502] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:15:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:15:12.678948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c839f7f-4302-4eff-9765-527dd189ecb5 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:12.679236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c839f7f-4302-4eff-9765-527dd189ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:12.679515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c839f7f-4302-4eff-9765-527dd189ecb5 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:12.680907 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 749/1503] 10.222.0.71 () {54 vars in 964 bytes} [Mon Jul 18 14:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:15:12.686640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028d76d-7f53-4480-8d42-0d0bfa21243f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:12.686844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028d76d-7f53-4480-8d42-0d0bfa21243f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:12.687127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028d76d-7f53-4480-8d42-0d0bfa21243f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:12.720918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:12.771772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f028d76d-7f53-4480-8d42-0d0bfa21243f None None] MFA Rules not processed for user `812ba8119a5a4297b409525861ec8118`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:12.773016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f028d76d-7f53-4480-8d42-0d0bfa21243f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:12.875069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 755/1504] 10.222.0.71 () {58 vars in 1061 bytes} [Mon Jul 18 14:15:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:15:12.883423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c839f7f-4302-4eff-9765-527dd189ecb5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:12.884964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2c839f7f-4302-4eff-9765-527dd189ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:12.950728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:13.016211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c839f7f-4302-4eff-9765-527dd189ecb5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:13.119209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:13.119983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:13.120379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:13.120803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:13.122800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:13.238433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:13.240151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:13.243728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:13.260376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 750/1505] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:15:13.311541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50e35-5295-4fb0-8d1b-f3d7b51e3d1d None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:13.311834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50e35-5295-4fb0-8d1b-f3d7b51e3d1d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:13.312112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc50e35-5295-4fb0-8d1b-f3d7b51e3d1d None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:13.313435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 756/1506] 10.222.0.71 () {52 vars in 829 bytes} [Mon Jul 18 14:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:15:13.317383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:13.318641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:13.431919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaa6b1-2c02-4d66-a0dd-75f7d5ad81e1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:13.540328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:13.541159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:13.541512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:13.542010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] PATH_INFO: `/v3/users/963c3fb0b8a64a3dbac7a36c55d16d04` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:13.558971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=963c3fb0b8a64a3dbac7a36c55d16d04)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:13.560322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:15:13.560322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jul 18 14:15:13.560322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: Jul 18 14:15:13.561589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:13.572998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:13.638195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963c3fb0b8a64a3dbac7a36c55d16d04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:13.649060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963c3fb0b8a64a3dbac7a36c55d16d04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:13.672112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963c3fb0b8a64a3dbac7a36c55d16d04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:13.672623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963c3fb0b8a64a3dbac7a36c55d16d04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:13.673174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963c3fb0b8a64a3dbac7a36c55d16d04'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:13.693189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 751/1507] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:15:13 2022] DELETE /identity/v3/users/963c3fb0b8a64a3dbac7a36c55d16d04 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:15:13.697620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55de84-305d-4e8b-a9af-ac6ba45ddeac None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:13.697923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55de84-305d-4e8b-a9af-ac6ba45ddeac None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:13.698354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55de84-305d-4e8b-a9af-ac6ba45ddeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:13.760771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9a55de84-305d-4e8b-a9af-ac6ba45ddeac None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:15:13.762867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9a55de84-305d-4e8b-a9af-ac6ba45ddeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:13.873782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:13.894464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 757/1508] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:15:13.906357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:13.907614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:14.029956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67cd9b1-87f8-402b-830c-cff4afacdc08 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:14.097912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:14.129065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3b6f9-14fa-423f-be1a-6bb4b5fab4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:14.130108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b6f9-14fa-423f-be1a-6bb4b5fab4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:14.130483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b6f9-14fa-423f-be1a-6bb4b5fab4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:14.130765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3b6f9-14fa-423f-be1a-6bb4b5fab4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:14.132542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c0f3b6f9-14fa-423f-be1a-6bb4b5fab4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:14.310195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b6f9-14fa-423f-be1a-6bb4b5fab4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:14.312443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f3b6f9-14fa-423f-be1a-6bb4b5fab4c7 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:14.337629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 752/1509] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:15:14.801441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55de84-305d-4e8b-a9af-ac6ba45ddeac None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:15:14.802897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9a55de84-305d-4e8b-a9af-ac6ba45ddeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:15:14.819958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55de84-305d-4e8b-a9af-ac6ba45ddeac None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:15:14.985834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:15:14.987289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:15:14.987800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:15:14.988287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] PATH_INFO: `/v3/projects/9be8edd85a844dcd895c5a04d229850d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:15:15.009494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be8edd85a844dcd895c5a04d229850d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:15:15.011013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:15:15.011013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jul 18 14:15:15.011013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: Jul 18 14:15:15.013055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:15:15.094740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be8edd85a844dcd895c5a04d229850d', 'request_id': 'req-8180353d-d992-444c-8a4e-088d11383fe7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:15.111697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be8edd85a844dcd895c5a04d229850d', 'request_id': 'req-8180353d-d992-444c-8a4e-088d11383fe7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:15.124400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be8edd85a844dcd895c5a04d229850d', 'request_id': 'req-8180353d-d992-444c-8a4e-088d11383fe7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:15.160090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] The token cache is being invalidate because project 9be8edd85a844dcd895c5a04d229850d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:15:15.160387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:15.160984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8180353d-d992-444c-8a4e-088d11383fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:15:15.162505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 758/1510] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:15:14 2022] DELETE /identity/v3/projects/9be8edd85a844dcd895c5a04d229850d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:15:24.334503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=62 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:15:25.157431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:16:59.617734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dce42e-17c3-4bcd-8674-0324dba8bad0 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:59.618358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dce42e-17c3-4bcd-8674-0324dba8bad0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:59.618358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dce42e-17c3-4bcd-8674-0324dba8bad0 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:59.619749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:16:59.626102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a9f16-1801-4cdb-a386-bbb3777c2f81 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:16:59.626860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a9f16-1801-4cdb-a386-bbb3777c2f81 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:16:59.627022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a9f16-1801-4cdb-a386-bbb3777c2f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:16:59.854929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9e0a9f16-1801-4cdb-a386-bbb3777c2f81 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:16:59.856733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9e0a9f16-1801-4cdb-a386-bbb3777c2f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:16:59.990346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:00.147863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dce42e-17c3-4bcd-8674-0324dba8bad0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:00.151179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-94dce42e-17c3-4bcd-8674-0324dba8bad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:00.292105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dce42e-17c3-4bcd-8674-0324dba8bad0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:00.297754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0923e2-9dfe-438c-ac86-42b89815a10e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:00.298421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0923e2-9dfe-438c-ac86-42b89815a10e service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:00.298707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0923e2-9dfe-438c-ac86-42b89815a10e service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:00.298985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0923e2-9dfe-438c-ac86-42b89815a10e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:00.307959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0923e2-9dfe-438c-ac86-42b89815a10e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:00.309602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0923e2-9dfe-438c-ac86-42b89815a10e service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:00.319168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 754/1513] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:01.177189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998e538-e265-4d09-8ea2-dda54f15bbd4 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:01.177790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998e538-e265-4d09-8ea2-dda54f15bbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:01.177790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998e538-e265-4d09-8ea2-dda54f15bbd4 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:01.179380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:17:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:17:01.184949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a6d0d-e30c-4753-9651-85e43917d93d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:01.185478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a6d0d-e30c-4753-9651-85e43917d93d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:01.185478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a6d0d-e30c-4753-9651-85e43917d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:01.208221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-033a6d0d-e30c-4753-9651-85e43917d93d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:01.209469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-033a6d0d-e30c-4753-9651-85e43917d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:01.219234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:01.379318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998e538-e265-4d09-8ea2-dda54f15bbd4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:01.395702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998e538-e265-4d09-8ea2-dda54f15bbd4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:01.402673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b17206-535e-4334-8cc7-d2720b7be49b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:01.403288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b17206-535e-4334-8cc7-d2720b7be49b service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:01.403610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b17206-535e-4334-8cc7-d2720b7be49b service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:01.403889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b17206-535e-4334-8cc7-d2720b7be49b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:01.413274 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b17206-535e-4334-8cc7-d2720b7be49b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:01.414940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b17206-535e-4334-8cc7-d2720b7be49b service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:01.425444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 761/1516] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:02.368316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5db188-9217-4325-856a-99d2a3cb0437 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:02.368924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5db188-9217-4325-856a-99d2a3cb0437 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:02.368924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5db188-9217-4325-856a-99d2a3cb0437 None None] PATH_INFO: `/v3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:02.370198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 756/1517] 10.222.0.104 () {54 vars in 926 bytes} [Mon Jul 18 14:17:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:17:02.377819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e33e3-417f-4690-8ba4-affe80b394b0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:02.378445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e33e3-417f-4690-8ba4-affe80b394b0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:02.378445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e33e3-417f-4690-8ba4-affe80b394b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:02.445926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f26e33e3-417f-4690-8ba4-affe80b394b0 None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:02.447155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f26e33e3-417f-4690-8ba4-affe80b394b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:02.466800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:17:02.540709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 762/1518] 10.222.0.104 () {58 vars in 1016 bytes} [Mon Jul 18 14:17:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:06.591733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05dd6-b409-4231-a708-0d93d25fe910 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:06.592621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05dd6-b409-4231-a708-0d93d25fe910 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:06.592621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a05dd6-b409-4231-a708-0d93d25fe910 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:06.594772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 757/1519] 10.222.0.71 () {54 vars in 961 bytes} [Mon Jul 18 14:17:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:17:06.602659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc2c97-688d-4164-909c-2d21830e39a0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:06.603153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc2c97-688d-4164-909c-2d21830e39a0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:06.603270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc2c97-688d-4164-909c-2d21830e39a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:06.622691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f0dc2c97-688d-4164-909c-2d21830e39a0 None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:06.623906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f0dc2c97-688d-4164-909c-2d21830e39a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:06.634503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 763/1520] 10.222.0.71 () {58 vars in 1058 bytes} [Mon Jul 18 14:17:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:06.642449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a05dd6-b409-4231-a708-0d93d25fe910 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:06.658577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a05dd6-b409-4231-a708-0d93d25fe910 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:06.664070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d339175-d616-4f9e-913f-328bbb665239 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:06.664723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339175-d616-4f9e-913f-328bbb665239 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:06.665009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339175-d616-4f9e-913f-328bbb665239 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:06.665287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339175-d616-4f9e-913f-328bbb665239 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:06.674997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d339175-d616-4f9e-913f-328bbb665239 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:06.677452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d339175-d616-4f9e-913f-328bbb665239 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:06.688607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 758/1521] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:12.660452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf05a1d-b98f-4d43-b4ab-4f60933d391f None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:12.661022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf05a1d-b98f-4d43-b4ab-4f60933d391f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:12.661022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf05a1d-b98f-4d43-b4ab-4f60933d391f None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:12.662514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:17:12.668204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5617a8-4e5f-4a16-9faf-4ce50e793a37 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:12.668671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5617a8-4e5f-4a16-9faf-4ce50e793a37 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:12.668787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5617a8-4e5f-4a16-9faf-4ce50e793a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:12.687477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-dd5617a8-4e5f-4a16-9faf-4ce50e793a37 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:12.688745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dd5617a8-4e5f-4a16-9faf-4ce50e793a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:12.697642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:12.853359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf05a1d-b98f-4d43-b4ab-4f60933d391f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:12.869995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf05a1d-b98f-4d43-b4ab-4f60933d391f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:12.876296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b697b93-e7f4-4ab1-b528-054c1be56f53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:12.876931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b697b93-e7f4-4ab1-b528-054c1be56f53 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:12.877213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b697b93-e7f4-4ab1-b528-054c1be56f53 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:12.877488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b697b93-e7f4-4ab1-b528-054c1be56f53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:12.886589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b697b93-e7f4-4ab1-b528-054c1be56f53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:12.888202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b697b93-e7f4-4ab1-b528-054c1be56f53 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:12.898370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 765/1524] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:13.736156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad783b-0568-40d2-bf48-c03906b176ee None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:13.736758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad783b-0568-40d2-bf48-c03906b176ee None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:13.736758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad783b-0568-40d2-bf48-c03906b176ee None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:13.738178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 18 14:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:17:13.743887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be9f9-82d5-4369-855f-0793d6f2d8cd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:13.744355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be9f9-82d5-4369-855f-0793d6f2d8cd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:13.744454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754be9f9-82d5-4369-855f-0793d6f2d8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:13.764615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-754be9f9-82d5-4369-855f-0793d6f2d8cd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:13.765902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-754be9f9-82d5-4369-855f-0793d6f2d8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:13.776277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 18 14:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:13.928189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad783b-0568-40d2-bf48-c03906b176ee None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:13.947706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad783b-0568-40d2-bf48-c03906b176ee None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:13.955361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323aaaf-928c-4ce8-bca4-2dd509ad2209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:13.956020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323aaaf-928c-4ce8-bca4-2dd509ad2209 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:13.956309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323aaaf-928c-4ce8-bca4-2dd509ad2209 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:13.956589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323aaaf-928c-4ce8-bca4-2dd509ad2209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:13.966791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323aaaf-928c-4ce8-bca4-2dd509ad2209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:13.969308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323aaaf-928c-4ce8-bca4-2dd509ad2209 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:13.981754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 761/1527] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:18.127324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ac9e-10cd-4fb4-904e-f654c3323778 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:18.127979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ac9e-10cd-4fb4-904e-f654c3323778 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:18.127979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40ac9e-10cd-4fb4-904e-f654c3323778 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:18.129270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:17:18.135437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1a188-3348-4faa-9e03-b9ab49842fa4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:18.135918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1a188-3348-4faa-9e03-b9ab49842fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:18.136044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1a188-3348-4faa-9e03-b9ab49842fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:18.155008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fdd1a188-3348-4faa-9e03-b9ab49842fa4 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:18.156223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fdd1a188-3348-4faa-9e03-b9ab49842fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:18.165455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:18.205090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ac9e-10cd-4fb4-904e-f654c3323778 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:18.207237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6c40ac9e-10cd-4fb4-904e-f654c3323778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:18.304928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40ac9e-10cd-4fb4-904e-f654c3323778 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:18.309228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abeb91-70ca-4011-af38-5276b592ab04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:18.309911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abeb91-70ca-4011-af38-5276b592ab04 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:18.310194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abeb91-70ca-4011-af38-5276b592ab04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:18.310467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abeb91-70ca-4011-af38-5276b592ab04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:18.320414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abeb91-70ca-4011-af38-5276b592ab04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:18.321897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abeb91-70ca-4011-af38-5276b592ab04 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:18.331838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 768/1530] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:20.375578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031a667-932b-4281-89cf-413b5f1557e7 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:20.376154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031a667-932b-4281-89cf-413b5f1557e7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:20.376154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031a667-932b-4281-89cf-413b5f1557e7 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:20.377494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:17:20.384157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10cbc-e3f9-4766-adbb-c8e0e9563293 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:20.384647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10cbc-e3f9-4766-adbb-c8e0e9563293 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:20.384752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc10cbc-e3f9-4766-adbb-c8e0e9563293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:20.408156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5fc10cbc-e3f9-4766-adbb-c8e0e9563293 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:20.409377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5fc10cbc-e3f9-4766-adbb-c8e0e9563293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:20.419148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:20.459301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031a667-932b-4281-89cf-413b5f1557e7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:20.474465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031a667-932b-4281-89cf-413b5f1557e7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:20.479917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eab1e-77e5-4728-8138-39d94773f6f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:20.480729 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eab1e-77e5-4728-8138-39d94773f6f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:20.481119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eab1e-77e5-4728-8138-39d94773f6f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:20.481505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eab1e-77e5-4728-8138-39d94773f6f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:20.493033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eab1e-77e5-4728-8138-39d94773f6f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:20.494507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eab1e-77e5-4728-8138-39d94773f6f6 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:20.504683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 764/1533] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:24.099146 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732aa307-4dee-42f1-838c-7bc8b1bafd45 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:24.099764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732aa307-4dee-42f1-838c-7bc8b1bafd45 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:24.099764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732aa307-4dee-42f1-838c-7bc8b1bafd45 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:24.101160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:17:24.106842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c198bba-44e1-4510-b784-7e2599c83ca7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:24.107324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c198bba-44e1-4510-b784-7e2599c83ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:24.107435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c198bba-44e1-4510-b784-7e2599c83ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:24.129849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8c198bba-44e1-4510-b784-7e2599c83ca7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:24.131067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8c198bba-44e1-4510-b784-7e2599c83ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:24.140180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:24.454388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732aa307-4dee-42f1-838c-7bc8b1bafd45 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:24.457010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-732aa307-4dee-42f1-838c-7bc8b1bafd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:24.528893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:17:24.543062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732aa307-4dee-42f1-838c-7bc8b1bafd45 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:24.546823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16b3ab-8079-4ef3-84d0-d5182bdeb183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:24.547469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16b3ab-8079-4ef3-84d0-d5182bdeb183 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:24.547751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16b3ab-8079-4ef3-84d0-d5182bdeb183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:24.548024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16b3ab-8079-4ef3-84d0-d5182bdeb183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:24.558125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16b3ab-8079-4ef3-84d0-d5182bdeb183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:24.559592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16b3ab-8079-4ef3-84d0-d5182bdeb183 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:24.570554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 771/1536] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:26.629611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a91b3-9aca-45cb-af11-3e9c1157d972 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:26.630540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a91b3-9aca-45cb-af11-3e9c1157d972 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:26.630540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a91b3-9aca-45cb-af11-3e9c1157d972 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:26.633188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 18 14:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:17:26.641400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fa903-07e7-470b-8c2b-6c776347f7c8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:26.641946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fa903-07e7-470b-8c2b-6c776347f7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:26.642302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fa903-07e7-470b-8c2b-6c776347f7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:26.664539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-369fa903-07e7-470b-8c2b-6c776347f7c8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:17:26.665825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-369fa903-07e7-470b-8c2b-6c776347f7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:26.675678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 18 14:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:17:26.987372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a91b3-9aca-45cb-af11-3e9c1157d972 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:17:26.990085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-243a91b3-9aca-45cb-af11-3e9c1157d972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:17:26.998656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:17:27.005070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a91b3-9aca-45cb-af11-3e9c1157d972 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:17:27.009457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cd1c6-5b42-43a9-a0e7-1432c2d9f2b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:17:27.010136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cd1c6-5b42-43a9-a0e7-1432c2d9f2b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:17:27.010421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cd1c6-5b42-43a9-a0e7-1432c2d9f2b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:17:27.010728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cd1c6-5b42-43a9-a0e7-1432c2d9f2b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:17:27.020145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cd1c6-5b42-43a9-a0e7-1432c2d9f2b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:17:27.021582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cd1c6-5b42-43a9-a0e7-1432c2d9f2b5 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:17:27.031860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 767/1539] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:17:36.659276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=11 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:17:37.027534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:58.117806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b65ba8-6dfc-4377-9812-7efe3512bccd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.118651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b65ba8-6dfc-4377-9812-7efe3512bccd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.118651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b65ba8-6dfc-4377-9812-7efe3512bccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.147714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f2b65ba8-6dfc-4377-9812-7efe3512bccd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.148996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f2b65ba8-6dfc-4377-9812-7efe3512bccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.164279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 773/1540] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.173603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d3cfb-5bd7-4623-aea8-eb6d6acd887a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.174240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d3cfb-5bd7-4623-aea8-eb6d6acd887a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.174240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d3cfb-5bd7-4623-aea8-eb6d6acd887a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.197232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f23d3cfb-5bd7-4623-aea8-eb6d6acd887a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.198570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f23d3cfb-5bd7-4623-aea8-eb6d6acd887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.213244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 768/1541] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.217815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b65ba8-6dfc-4377-9812-7efe3512bccd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.231632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b65ba8-6dfc-4377-9812-7efe3512bccd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:58.244314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93eef-b944-4335-b9e0-e237aea42312 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.244693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93eef-b944-4335-b9e0-e237aea42312 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.244894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e93eef-b944-4335-b9e0-e237aea42312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.252397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b897fca-a75b-42a0-bec5-c0c05e3bb892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:58.253141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b897fca-a75b-42a0-bec5-c0c05e3bb892 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.253424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b897fca-a75b-42a0-bec5-c0c05e3bb892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.259991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b897fca-a75b-42a0-bec5-c0c05e3bb892 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.261277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b897fca-a75b-42a0-bec5-c0c05e3bb892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:18:58.261602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b897fca-a75b-42a0-bec5-c0c05e3bb892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:58.267767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b897fca-a75b-42a0-bec5-c0c05e3bb892 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:58.287724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b7e93eef-b944-4335-b9e0-e237aea42312 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.288977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b7e93eef-b944-4335-b9e0-e237aea42312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.296558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 774/1542] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:18:58.297739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70837e9b-a6ac-48e6-86b8-6f4599640cc5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.298029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70837e9b-a6ac-48e6-86b8-6f4599640cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.298310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70837e9b-a6ac-48e6-86b8-6f4599640cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.310568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 769/1543] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:18:58.311458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e93eef-b944-4335-b9e0-e237aea42312 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.334187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e93eef-b944-4335-b9e0-e237aea42312 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:58.335649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-70837e9b-a6ac-48e6-86b8-6f4599640cc5 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.336874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-70837e9b-a6ac-48e6-86b8-6f4599640cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.347589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e0f37-b874-4978-885f-6db9ffc9104c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:58.348653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e0f37-b874-4978-885f-6db9ffc9104c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.348867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e0f37-b874-4978-885f-6db9ffc9104c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.349122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e0f37-b874-4978-885f-6db9ffc9104c admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.349866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e0f37-b874-4978-885f-6db9ffc9104c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:58.353532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e0f37-b874-4978-885f-6db9ffc9104c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:58.359506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 775/1544] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.362797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff3661-91cf-4a42-9a35-eaa89eed156a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.363096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff3661-91cf-4a42-9a35-eaa89eed156a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.363485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff3661-91cf-4a42-9a35-eaa89eed156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.405278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-82ff3661-91cf-4a42-9a35-eaa89eed156a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.406542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-82ff3661-91cf-4a42-9a35-eaa89eed156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.430213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 776/1545] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.433757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 770/1546] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:58.434304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cc56b-c293-4dd7-870b-127647e0896c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.435678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e0f37-b874-4978-885f-6db9ffc9104c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.436125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cc56b-c293-4dd7-870b-127647e0896c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.436464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cc56b-c293-4dd7-870b-127647e0896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.475096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ee4cc56b-c293-4dd7-870b-127647e0896c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.476754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ee4cc56b-c293-4dd7-870b-127647e0896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.574191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e0f37-b874-4978-885f-6db9ffc9104c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:58.584900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01dc12-4c5c-4d97-9d44-f66e59ea8312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:58.585796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01dc12-4c5c-4d97-9d44-f66e59ea8312 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.586083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01dc12-4c5c-4d97-9d44-f66e59ea8312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.586361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01dc12-4c5c-4d97-9d44-f66e59ea8312 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.587105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01dc12-4c5c-4d97-9d44-f66e59ea8312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:18:58.587568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01dc12-4c5c-4d97-9d44-f66e59ea8312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:58.588952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-da01dc12-4c5c-4d97-9d44-f66e59ea8312 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:18:58.588952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jul 18 14:18:58.588952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: Jul 18 14:18:58.590203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01dc12-4c5c-4d97-9d44-f66e59ea8312 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:58.602204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 777/1547] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.603119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cc56b-c293-4dd7-870b-127647e0896c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.606130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 771/1548] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:18:58.607706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f66b-af18-44c3-8bb0-3728f237ef53 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.608236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f66b-af18-44c3-8bb0-3728f237ef53 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.608742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11f66b-af18-44c3-8bb0-3728f237ef53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.625834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cc56b-c293-4dd7-870b-127647e0896c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:58.639010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4a11f66b-af18-44c3-8bb0-3728f237ef53 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.640252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4a11f66b-af18-44c3-8bb0-3728f237ef53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.640795 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01057e-cb53-4aec-a041-5836b6bf3585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:58.642112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01057e-cb53-4aec-a041-5836b6bf3585 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.642385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01057e-cb53-4aec-a041-5836b6bf3585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.642679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01057e-cb53-4aec-a041-5836b6bf3585 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.643383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01057e-cb53-4aec-a041-5836b6bf3585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:58.648221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01057e-cb53-4aec-a041-5836b6bf3585 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:58.655193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 772/1549] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.657156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca46b39-70ac-4741-98cc-669d5a8af45b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.657455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca46b39-70ac-4741-98cc-669d5a8af45b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.657774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca46b39-70ac-4741-98cc-669d5a8af45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.672390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 778/1550] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/users => generated 293 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:58.673750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28278330-216e-41f6-b5c3-5677dc822dd6 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.673750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28278330-216e-41f6-b5c3-5677dc822dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.674137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28278330-216e-41f6-b5c3-5677dc822dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.680014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9ca46b39-70ac-4741-98cc-669d5a8af45b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.681303 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9ca46b39-70ac-4741-98cc-669d5a8af45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.689529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 773/1551] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.690395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca46b39-70ac-4741-98cc-669d5a8af45b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.693635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-28278330-216e-41f6-b5c3-5677dc822dd6 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.695264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-28278330-216e-41f6-b5c3-5677dc822dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.700271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca46b39-70ac-4741-98cc-669d5a8af45b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:58.703601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 779/1552] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.704888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a6c4-9d9c-4d83-9f7e-ddd84bc8ffe2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.705178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a6c4-9d9c-4d83-9f7e-ddd84bc8ffe2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.705441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:58.705573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589a6c4-9d9c-4d83-9f7e-ddd84bc8ffe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.706260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.706547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.706823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.707617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:18:58.707924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:58.709544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:58.722859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 774/1553] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:18:58.723733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.724935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9589a6c4-9d9c-4d83-9f7e-ddd84bc8ffe2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.726171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9589a6c4-9d9c-4d83-9f7e-ddd84bc8ffe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.734306 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 780/1554] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.742701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecd1d4-b90f-42e2-b18c-f7ebdd163e72 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:58.742819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a73f0e-4f92-4a64-8e73-5411bf67b11d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.742819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a73f0e-4f92-4a64-8e73-5411bf67b11d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.742819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a73f0e-4f92-4a64-8e73-5411bf67b11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.743086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f048fe-bc80-4001-afcd-6a7ed8519c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:58.743749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f048fe-bc80-4001-afcd-6a7ed8519c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.744228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f048fe-bc80-4001-afcd-6a7ed8519c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.744593 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f048fe-bc80-4001-afcd-6a7ed8519c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.745424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f048fe-bc80-4001-afcd-6a7ed8519c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:58.747466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f048fe-bc80-4001-afcd-6a7ed8519c9a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:58.756830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-40a73f0e-4f92-4a64-8e73-5411bf67b11d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.758069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-40a73f0e-4f92-4a64-8e73-5411bf67b11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.767285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 781/1555] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.768907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210657d-cdb0-4580-bcb8-f308cfb02280 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.769195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210657d-cdb0-4580-bcb8-f308cfb02280 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.769501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210657d-cdb0-4580-bcb8-f308cfb02280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.781955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 775/1556] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:58.782806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f048fe-bc80-4001-afcd-6a7ed8519c9a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.789238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9210657d-cdb0-4580-bcb8-f308cfb02280 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.790490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9210657d-cdb0-4580-bcb8-f308cfb02280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.851453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f048fe-bc80-4001-afcd-6a7ed8519c9a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:58.853074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 782/1557] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.854193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ad35-fe42-49ea-9fb8-37edcf3f6d1c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.854433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ad35-fe42-49ea-9fb8-37edcf3f6d1c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.854698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4ad35-fe42-49ea-9fb8-37edcf3f6d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.856956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b0173-73bf-47e8-a3f0-bfc1cf308ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:58.857746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b0173-73bf-47e8-a3f0-bfc1cf308ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.858022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b0173-73bf-47e8-a3f0-bfc1cf308ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.858441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b0173-73bf-47e8-a3f0-bfc1cf308ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.859222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b0173-73bf-47e8-a3f0-bfc1cf308ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:58.860866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b0173-73bf-47e8-a3f0-bfc1cf308ac1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:58.863265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:58.876057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-eff4ad35-fe42-49ea-9fb8-37edcf3f6d1c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.877277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-eff4ad35-fe42-49ea-9fb8-37edcf3f6d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.877617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 776/1558] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:18:58.879085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b1c2-5890-4ce3-8fe5-d23d80c4ff39 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.879432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b1c2-5890-4ce3-8fe5-d23d80c4ff39 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.879710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b1c2-5890-4ce3-8fe5-d23d80c4ff39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.885717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 783/1559] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.886514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4ad35-fe42-49ea-9fb8-37edcf3f6d1c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.895940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4ad35-fe42-49ea-9fb8-37edcf3f6d1c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:58.899487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5111b1c2-5890-4ce3-8fe5-d23d80c4ff39 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.900534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:58.900787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5111b1c2-5890-4ce3-8fe5-d23d80c4ff39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.901186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.901467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.901763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.902451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:58.903476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:18:58.903476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jul 18 14:18:58.903476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: Jul 18 14:18:58.904676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:58.909797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 777/1560] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:58.914435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e15a7-4c2a-4d9d-a5a9-c6c18fd1f16e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:58.914435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e15a7-4c2a-4d9d-a5a9-c6c18fd1f16e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:58.914435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e15a7-4c2a-4d9d-a5a9-c6c18fd1f16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:58.932359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e80e15a7-4c2a-4d9d-a5a9-c6c18fd1f16e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:58.933586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e80e15a7-4c2a-4d9d-a5a9-c6c18fd1f16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:58.937090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 784/1561] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:58.937090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:58.939465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:59.002368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 778/1562] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:59.003261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e15a7-4c2a-4d9d-a5a9-c6c18fd1f16e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.010755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f4dc5-c8dc-43c6-a4d5-dcf74f7adfca admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.016029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb3335-ba48-45ab-b1d1-6f1b8cffb273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.016747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb3335-ba48-45ab-b1d1-6f1b8cffb273 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.017029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb3335-ba48-45ab-b1d1-6f1b8cffb273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.017190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e15a7-4c2a-4d9d-a5a9-c6c18fd1f16e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.017296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb3335-ba48-45ab-b1d1-6f1b8cffb273 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.018080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb3335-ba48-45ab-b1d1-6f1b8cffb273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:18:59.018389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb3335-ba48-45ab-b1d1-6f1b8cffb273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.020228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdb3335-ba48-45ab-b1d1-6f1b8cffb273 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.022982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.023785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.024067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.024342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.025074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.026149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:18:59.026149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jul 18 14:18:59.026149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: Jul 18 14:18:59.027657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.034907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 785/1563] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:18:59.036455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543d12-ecdf-476c-82b7-cba9218823d9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.036738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543d12-ecdf-476c-82b7-cba9218823d9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.037014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28543d12-ecdf-476c-82b7-cba9218823d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.060758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-28543d12-ecdf-476c-82b7-cba9218823d9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:59.061010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 779/1564] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/users => generated 292 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.061010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.061214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-28543d12-ecdf-476c-82b7-cba9218823d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:59.069592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda1f8a-60d5-45f3-adcd-24f9a1b1ba21 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.070037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 786/1565] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:59.070896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28543d12-ecdf-476c-82b7-cba9218823d9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.080731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753ffd6-a2a5-4ce8-aac1-2f6c3bb84ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.081197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28543d12-ecdf-476c-82b7-cba9218823d9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.081588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ffd6-a2a5-4ce8-aac1-2f6c3bb84ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.082071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ffd6-a2a5-4ce8-aac1-2f6c3bb84ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.082432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753ffd6-a2a5-4ce8-aac1-2f6c3bb84ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.083376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753ffd6-a2a5-4ce8-aac1-2f6c3bb84ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.085406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753ffd6-a2a5-4ce8-aac1-2f6c3bb84ab4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.086981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.087654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.087939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.088216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.088958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:18:59.089266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.090959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.099831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 780/1566] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:18:59.100566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753ffd6-a2a5-4ce8-aac1-2f6c3bb84ab4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.106225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 787/1567] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:18:59.107747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.120448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4a788-13bc-416f-995e-ab0b3f21f07f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.121615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753ffd6-a2a5-4ce8-aac1-2f6c3bb84ab4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.125493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b4bae-9e26-467c-9293-5e3a96627db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.126202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b4bae-9e26-467c-9293-5e3a96627db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.126485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b4bae-9e26-467c-9293-5e3a96627db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.126758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b4bae-9e26-467c-9293-5e3a96627db1 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.127450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b4bae-9e26-467c-9293-5e3a96627db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.129628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b4bae-9e26-467c-9293-5e3a96627db1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.130471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.131644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.132162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.132672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.134092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:18:59.134668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.137804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.150831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 788/1568] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.155015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b4bae-9e26-467c-9293-5e3a96627db1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.160429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 781/1569] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:18:59.161989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.166882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b4bae-9e26-467c-9293-5e3a96627db1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.172871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.173547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.173868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.174139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.174859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:18:59.175227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.176779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b431b5-8412-4665-ad09-b0d58170cacc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.177588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.184299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10187660-0fd5-46ad-89f1-642937784097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.185249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10187660-0fd5-46ad-89f1-642937784097 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.185796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10187660-0fd5-46ad-89f1-642937784097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.186309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10187660-0fd5-46ad-89f1-642937784097 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.187799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10187660-0fd5-46ad-89f1-642937784097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.190221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 789/1570] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:18:59.191073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.191608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10187660-0fd5-46ad-89f1-642937784097 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.203738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b76dc8-1c67-43e4-8616-f08341e3a6b3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.209273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63820a85-af0b-4c55-b40f-6014fc02348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.210079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63820a85-af0b-4c55-b40f-6014fc02348f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.210296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63820a85-af0b-4c55-b40f-6014fc02348f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.210573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63820a85-af0b-4c55-b40f-6014fc02348f admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.211488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63820a85-af0b-4c55-b40f-6014fc02348f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.213267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63820a85-af0b-4c55-b40f-6014fc02348f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.222895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 782/1571] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.223776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10187660-0fd5-46ad-89f1-642937784097 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.224589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 790/1572] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:18:59.226525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63820a85-af0b-4c55-b40f-6014fc02348f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.238228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63820a85-af0b-4c55-b40f-6014fc02348f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.292858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10187660-0fd5-46ad-89f1-642937784097 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.298787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.299532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.299967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.300245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.300978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:18:59.301283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.302996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.315990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 783/1573] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:18:59.317710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.318913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.319972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.320253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.320538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] PATH_INFO: `/v3/projects/94c7507618b247c98e6bb541a48dbd31/users/a0210f69010b4529822bcdac1bdf8b8a/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.335281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ea47a-f25b-4d77-99b1-1bac81dd1950 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.337688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94c7507618b247c98e6bb541a48dbd31, user_id=a0210f69010b4529822bcdac1bdf8b8a, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.338796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:18:59.338796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jul 18 14:18:59.338796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: Jul 18 14:18:59.339938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12803b5-f8f7-4b48-92dd-4eee095be1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.340638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12803b5-f8f7-4b48-92dd-4eee095be1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.340818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.340925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12803b5-f8f7-4b48-92dd-4eee095be1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.341178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12803b5-f8f7-4b48-92dd-4eee095be1d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.341900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12803b5-f8f7-4b48-92dd-4eee095be1d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.343794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12803b5-f8f7-4b48-92dd-4eee095be1d9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.356664 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 791/1574] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:18:59 2022] PUT /identity/v3/projects/94c7507618b247c98e6bb541a48dbd31/users/a0210f69010b4529822bcdac1bdf8b8a/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:18:59.357956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.368162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:59.378231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 784/1575] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.379902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12803b5-f8f7-4b48-92dd-4eee095be1d9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.384547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:59.434590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81286cce-81c0-4ecd-8060-83a1d7d07d55 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.456986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12803b5-f8f7-4b48-92dd-4eee095be1d9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.462298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5d85-e88e-4510-9335-fc8b45a7e6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.463337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5d85-e88e-4510-9335-fc8b45a7e6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.463337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5d85-e88e-4510-9335-fc8b45a7e6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.463559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5d85-e88e-4510-9335-fc8b45a7e6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.464252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f5d85-e88e-4510-9335-fc8b45a7e6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.466222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f5d85-e88e-4510-9335-fc8b45a7e6d1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.494999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 785/1576] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.498572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5d85-e88e-4510-9335-fc8b45a7e6d1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.501990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b773f-4387-42a1-a9db-769f7f18c30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.502956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b773f-4387-42a1-a9db-769f7f18c30c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.503271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b773f-4387-42a1-a9db-769f7f18c30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.503620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b773f-4387-42a1-a9db-769f7f18c30c admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.505010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b773f-4387-42a1-a9db-769f7f18c30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.506854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b773f-4387-42a1-a9db-769f7f18c30c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.518201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 792/1577] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:18:59.522118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b773f-4387-42a1-a9db-769f7f18c30c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.532828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b773f-4387-42a1-a9db-769f7f18c30c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.578181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5d85-e88e-4510-9335-fc8b45a7e6d1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.584268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cff80-5127-4684-adfc-43b788f7a94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.584949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cff80-5127-4684-adfc-43b788f7a94d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.585234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cff80-5127-4684-adfc-43b788f7a94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.585509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232cff80-5127-4684-adfc-43b788f7a94d admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.586260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cff80-5127-4684-adfc-43b788f7a94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.589173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232cff80-5127-4684-adfc-43b788f7a94d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.603848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a30fa2-7574-4ecc-bcf6-b7ef27afa871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.604544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a30fa2-7574-4ecc-bcf6-b7ef27afa871 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.604857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a30fa2-7574-4ecc-bcf6-b7ef27afa871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.605136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a30fa2-7574-4ecc-bcf6-b7ef27afa871 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.605877 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a30fa2-7574-4ecc-bcf6-b7ef27afa871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.618339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a30fa2-7574-4ecc-bcf6-b7ef27afa871 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.618784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 786/1578] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.619695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cff80-5127-4684-adfc-43b788f7a94d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.649287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 793/1579] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.651094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a30fa2-7574-4ecc-bcf6-b7ef27afa871 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.663099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a30fa2-7574-4ecc-bcf6-b7ef27afa871 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.704387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232cff80-5127-4684-adfc-43b788f7a94d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.710309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdff83-0431-4a29-86c0-ca8f8d7bcf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.710982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdff83-0431-4a29-86c0-ca8f8d7bcf17 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.711268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdff83-0431-4a29-86c0-ca8f8d7bcf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.711547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdff83-0431-4a29-86c0-ca8f8d7bcf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.712353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdff83-0431-4a29-86c0-ca8f8d7bcf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.714038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdff83-0431-4a29-86c0-ca8f8d7bcf17 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.725191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 787/1580] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:18:59.726601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702728a9-417a-431a-b883-d63db1418b82 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.726896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702728a9-417a-431a-b883-d63db1418b82 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.727179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702728a9-417a-431a-b883-d63db1418b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.740320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:59.747334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:18:59.753485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaeeed-3b8c-4765-8227-a8b6a4239641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.754352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaeeed-3b8c-4765-8227-a8b6a4239641 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.754898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-702728a9-417a-431a-b883-d63db1418b82 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:18:59.755019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaeeed-3b8c-4765-8227-a8b6a4239641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.755019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaeeed-3b8c-4765-8227-a8b6a4239641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.755604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaeeed-3b8c-4765-8227-a8b6a4239641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.756061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-702728a9-417a-431a-b883-d63db1418b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:18:59.757577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaeeed-3b8c-4765-8227-a8b6a4239641 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.764886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 788/1581] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:18:59.769279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702728a9-417a-431a-b883-d63db1418b82 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.781392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702728a9-417a-431a-b883-d63db1418b82 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.787311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6cf64-ce3b-4974-bb7a-928f1fc3b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.788002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6cf64-ce3b-4974-bb7a-928f1fc3b59b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.788289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6cf64-ce3b-4974-bb7a-928f1fc3b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.788563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6cf64-ce3b-4974-bb7a-928f1fc3b59b admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.792242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 794/1582] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.792424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6cf64-ce3b-4974-bb7a-928f1fc3b59b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.792526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaeeed-3b8c-4765-8227-a8b6a4239641 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.792609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6cf64-ce3b-4974-bb7a-928f1fc3b59b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.813767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 789/1583] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/users => generated 286 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.814439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6cf64-ce3b-4974-bb7a-928f1fc3b59b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.826508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6cf64-ce3b-4974-bb7a-928f1fc3b59b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.865301 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaeeed-3b8c-4765-8227-a8b6a4239641 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.870466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbf522-8f7c-4b0d-932d-c408be1af5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.871081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbf522-8f7c-4b0d-932d-c408be1af5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.871358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbf522-8f7c-4b0d-932d-c408be1af5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.871635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbf522-8f7c-4b0d-932d-c408be1af5c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.872343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbf522-8f7c-4b0d-932d-c408be1af5c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.874572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbf522-8f7c-4b0d-932d-c408be1af5c3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.901497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf504c4-9177-4a72-9958-cb9a4f987ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.902291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf504c4-9177-4a72-9958-cb9a4f987ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.902636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf504c4-9177-4a72-9958-cb9a4f987ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.902878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf504c4-9177-4a72-9958-cb9a4f987ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.903651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf504c4-9177-4a72-9958-cb9a4f987ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.904449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 795/1584] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.905597 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf504c4-9177-4a72-9958-cb9a4f987ed3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.905856 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbf522-8f7c-4b0d-932d-c408be1af5c3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.917633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbf522-8f7c-4b0d-932d-c408be1af5c3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.921267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 790/1585] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:18:59.921267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf504c4-9177-4a72-9958-cb9a4f987ed3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.925203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf4641-2624-4834-bb47-4b9787b40786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.925989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4641-2624-4834-bb47-4b9787b40786 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.926246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4641-2624-4834-bb47-4b9787b40786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.926664 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf4641-2624-4834-bb47-4b9787b40786 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.927422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf4641-2624-4834-bb47-4b9787b40786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.929420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf4641-2624-4834-bb47-4b9787b40786 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.933082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf504c4-9177-4a72-9958-cb9a4f987ed3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.937989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99680ed-e80a-4c9e-8926-b4c07c27f173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.938706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99680ed-e80a-4c9e-8926-b4c07c27f173 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.938988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99680ed-e80a-4c9e-8926-b4c07c27f173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.939265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99680ed-e80a-4c9e-8926-b4c07c27f173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:18:59.940332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99680ed-e80a-4c9e-8926-b4c07c27f173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:18:59.942445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99680ed-e80a-4c9e-8926-b4c07c27f173 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:18:59.954529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 791/1586] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:18:59.959585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 796/1587] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:18:59 2022] POST /identity/v3/users => generated 296 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:18:59.959862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99680ed-e80a-4c9e-8926-b4c07c27f173 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.959958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf4641-2624-4834-bb47-4b9787b40786 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:18:59.968965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99680ed-e80a-4c9e-8926-b4c07c27f173 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:18:59.974960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72deeeb-dbea-4b15-82ae-2d0b76b8614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:18:59.976009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72deeeb-dbea-4b15-82ae-2d0b76b8614c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:18:59.976419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72deeeb-dbea-4b15-82ae-2d0b76b8614c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:18:59.976820 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72deeeb-dbea-4b15-82ae-2d0b76b8614c admin admin] PATH_INFO: `/v3/projects/c64ef5c9e4e74877afbe0b977348706e/users/93f4adb167914e1b82542cbd87f3f214/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.000449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72deeeb-dbea-4b15-82ae-2d0b76b8614c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64ef5c9e4e74877afbe0b977348706e, user_id=93f4adb167914e1b82542cbd87f3f214, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.005978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72deeeb-dbea-4b15-82ae-2d0b76b8614c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.024944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 792/1588] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:18:59 2022] PUT /identity/v3/projects/c64ef5c9e4e74877afbe0b977348706e/users/93f4adb167914e1b82542cbd87f3f214/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:00.026155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72deeeb-dbea-4b15-82ae-2d0b76b8614c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.047005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf4641-2624-4834-bb47-4b9787b40786 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.099844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72deeeb-dbea-4b15-82ae-2d0b76b8614c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.100856 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642a61-23ad-4e14-ada2-80319e1d95bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.101561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642a61-23ad-4e14-ada2-80319e1d95bd service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.101925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642a61-23ad-4e14-ada2-80319e1d95bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.102252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642a61-23ad-4e14-ada2-80319e1d95bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.105816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8755b6-2772-41db-88f8-3aaec7aa47a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.106662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8755b6-2772-41db-88f8-3aaec7aa47a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.107226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8755b6-2772-41db-88f8-3aaec7aa47a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.107497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8755b6-2772-41db-88f8-3aaec7aa47a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.108206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8755b6-2772-41db-88f8-3aaec7aa47a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.110253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8755b6-2772-41db-88f8-3aaec7aa47a4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.112371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642a61-23ad-4e14-ada2-80319e1d95bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.113944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642a61-23ad-4e14-ada2-80319e1d95bd service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.125443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 797/1589] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:00.126380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642a61-23ad-4e14-ada2-80319e1d95bd service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.136849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 793/1590] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:00 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:00.137310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642a61-23ad-4e14-ada2-80319e1d95bd service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.137739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8755b6-2772-41db-88f8-3aaec7aa47a4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.143306 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4beefe-22e0-48d0-9192-cb7d71a126d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.144921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4beefe-22e0-48d0-9192-cb7d71a126d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.144921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4beefe-22e0-48d0-9192-cb7d71a126d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.144921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4beefe-22e0-48d0-9192-cb7d71a126d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.146009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4beefe-22e0-48d0-9192-cb7d71a126d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.147663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4beefe-22e0-48d0-9192-cb7d71a126d8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.149033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8755b6-2772-41db-88f8-3aaec7aa47a4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.149431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:00.159377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b06f0d-dc7b-4bb1-9f46-80f7db42d21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.160090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b06f0d-dc7b-4bb1-9f46-80f7db42d21a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.161423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b06f0d-dc7b-4bb1-9f46-80f7db42d21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.161423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b06f0d-dc7b-4bb1-9f46-80f7db42d21a admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.162076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b06f0d-dc7b-4bb1-9f46-80f7db42d21a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.162309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 798/1591] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.163186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4beefe-22e0-48d0-9192-cb7d71a126d8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.164460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b06f0d-dc7b-4bb1-9f46-80f7db42d21a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.176038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 794/1592] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.176638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4beefe-22e0-48d0-9192-cb7d71a126d8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.177190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b06f0d-dc7b-4bb1-9f46-80f7db42d21a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.184167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90c1e7-8c87-42bb-be49-f0c82dce547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.185072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90c1e7-8c87-42bb-be49-f0c82dce547d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.185342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90c1e7-8c87-42bb-be49-f0c82dce547d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.185608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90c1e7-8c87-42bb-be49-f0c82dce547d admin admin] PATH_INFO: `/v3/projects/8788b8d25f234df6895ea3f8fa046610/users/2e3c7eb6a5cb4515b6f48520535fea76/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.187629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b06f0d-dc7b-4bb1-9f46-80f7db42d21a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.197917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa346dd-06ce-4cd5-9a5e-b87b10904cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.198739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa346dd-06ce-4cd5-9a5e-b87b10904cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.199064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa346dd-06ce-4cd5-9a5e-b87b10904cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.199407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa346dd-06ce-4cd5-9a5e-b87b10904cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.200279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa346dd-06ce-4cd5-9a5e-b87b10904cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.202246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa346dd-06ce-4cd5-9a5e-b87b10904cc0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.203195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90c1e7-8c87-42bb-be49-f0c82dce547d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8788b8d25f234df6895ea3f8fa046610, user_id=2e3c7eb6a5cb4515b6f48520535fea76, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.206116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90c1e7-8c87-42bb-be49-f0c82dce547d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.216113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 795/1593] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.216909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa346dd-06ce-4cd5-9a5e-b87b10904cc0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.225257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 799/1594] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:00 2022] PUT /identity/v3/projects/8788b8d25f234df6895ea3f8fa046610/users/2e3c7eb6a5cb4515b6f48520535fea76/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:00.227659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf72e92-358d-44a1-9741-1f80c5758b48 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.228182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf72e92-358d-44a1-9741-1f80c5758b48 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.229169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa346dd-06ce-4cd5-9a5e-b87b10904cc0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.230108 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf72e92-358d-44a1-9741-1f80c5758b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.238534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:00.261493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8bf72e92-358d-44a1-9741-1f80c5758b48 None None] MFA Rules not processed for user `93f4adb167914e1b82542cbd87f3f214`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:00.263174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8bf72e92-358d-44a1-9741-1f80c5758b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:00.299214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126122f-d736-4bba-ba8f-c30aed8de3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.300086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126122f-d736-4bba-ba8f-c30aed8de3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.300476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126122f-d736-4bba-ba8f-c30aed8de3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.301458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126122f-d736-4bba-ba8f-c30aed8de3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.302453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126122f-d736-4bba-ba8f-c30aed8de3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.304102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126122f-d736-4bba-ba8f-c30aed8de3b4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.316106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 796/1595] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.317749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126122f-d736-4bba-ba8f-c30aed8de3b4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.334043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126122f-d736-4bba-ba8f-c30aed8de3b4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.345719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576950-6d5c-4529-b03e-5bc78be0461c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.346491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576950-6d5c-4529-b03e-5bc78be0461c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.346846 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576950-6d5c-4529-b03e-5bc78be0461c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.347227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f576950-6d5c-4529-b03e-5bc78be0461c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.348247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f576950-6d5c-4529-b03e-5bc78be0461c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.350023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f576950-6d5c-4529-b03e-5bc78be0461c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.360762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 800/1596] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:00.361536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 797/1597] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.362405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf72e92-358d-44a1-9741-1f80c5758b48 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.362572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576950-6d5c-4529-b03e-5bc78be0461c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.376384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf72e92-358d-44a1-9741-1f80c5758b48 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.381223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f576950-6d5c-4529-b03e-5bc78be0461c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.384228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd240db3-9919-4a00-87af-5e8f9ec51f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.385643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240db3-9919-4a00-87af-5e8f9ec51f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.386133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240db3-9919-4a00-87af-5e8f9ec51f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.386626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240db3-9919-4a00-87af-5e8f9ec51f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.386956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22995c71-6133-4588-bdf0-4b240edf6f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.387734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240db3-9919-4a00-87af-5e8f9ec51f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.388156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995c71-6133-4588-bdf0-4b240edf6f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.388542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995c71-6133-4588-bdf0-4b240edf6f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.388925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995c71-6133-4588-bdf0-4b240edf6f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.389860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22995c71-6133-4588-bdf0-4b240edf6f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.390251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240db3-9919-4a00-87af-5e8f9ec51f69 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.391679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22995c71-6133-4588-bdf0-4b240edf6f23 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.405328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 798/1598] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.406401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 801/1599] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.406653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22995c71-6133-4588-bdf0-4b240edf6f23 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.408430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aae07-82e5-497e-9312-a1342e92667b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.408923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aae07-82e5-497e-9312-a1342e92667b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.409445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49aae07-82e5-497e-9312-a1342e92667b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.424952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22995c71-6133-4588-bdf0-4b240edf6f23 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.432085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fc04f-fefa-44f6-a3cf-df489f86f3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.432498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e49aae07-82e5-497e-9312-a1342e92667b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:00.433180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc04f-fefa-44f6-a3cf-df489f86f3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.433529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc04f-fefa-44f6-a3cf-df489f86f3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.433914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fc04f-fefa-44f6-a3cf-df489f86f3ac admin admin] PATH_INFO: `/v3/projects/b2c6da087ce74a1d8ef8cbad1d1506ae/users/4234cd5ae2484fcc85c1e73b09af992b/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.434429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e49aae07-82e5-497e-9312-a1342e92667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:00.445466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 802/1600] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:00.446866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aae07-82e5-497e-9312-a1342e92667b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.455937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc04f-fefa-44f6-a3cf-df489f86f3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c6da087ce74a1d8ef8cbad1d1506ae, user_id=4234cd5ae2484fcc85c1e73b09af992b, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.459524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fc04f-fefa-44f6-a3cf-df489f86f3ac admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.459709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49aae07-82e5-497e-9312-a1342e92667b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.465172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a596a5-8b11-4249-8a2f-195737bf8253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.466111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a596a5-8b11-4249-8a2f-195737bf8253 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.466389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a596a5-8b11-4249-8a2f-195737bf8253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.466660 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a596a5-8b11-4249-8a2f-195737bf8253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.467403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a596a5-8b11-4249-8a2f-195737bf8253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.469121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a596a5-8b11-4249-8a2f-195737bf8253 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.484061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 803/1602] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.484061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a596a5-8b11-4249-8a2f-195737bf8253 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.484412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 799/1602] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:00 2022] PUT /identity/v3/projects/b2c6da087ce74a1d8ef8cbad1d1506ae/users/4234cd5ae2484fcc85c1e73b09af992b/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:00.486888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e845e5-2476-45b5-bb22-71931ec5aaed None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.487394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e845e5-2476-45b5-bb22-71931ec5aaed None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.487681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e845e5-2476-45b5-bb22-71931ec5aaed None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.494012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 800/1603] 10.222.0.71 () {54 vars in 962 bytes} [Mon Jul 18 14:19:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:00.495122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e845e5-2476-45b5-bb22-71931ec5aaed None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.511932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a596a5-8b11-4249-8a2f-195737bf8253 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.580497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e845e5-2476-45b5-bb22-71931ec5aaed None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.588116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.589780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.589780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.590030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] PATH_INFO: `/v3/projects/e1166ac77e8e403aa5f81e452bc039ac/users/5ca8be8093b547ee953ea646f90fdcae/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.596399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a41e8-c52d-42e1-a4b2-8d7f4ff11612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.597005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a41e8-c52d-42e1-a4b2-8d7f4ff11612 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.597392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a41e8-c52d-42e1-a4b2-8d7f4ff11612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.597896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a41e8-c52d-42e1-a4b2-8d7f4ff11612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.599642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a41e8-c52d-42e1-a4b2-8d7f4ff11612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.602228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a41e8-c52d-42e1-a4b2-8d7f4ff11612 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.613914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1166ac77e8e403aa5f81e452bc039ac, user_id=5ca8be8093b547ee953ea646f90fdcae, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.615995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 804/1604] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:00.616921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a41e8-c52d-42e1-a4b2-8d7f4ff11612 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.618203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.632702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a41e8-c52d-42e1-a4b2-8d7f4ff11612 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.643138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 801/1605] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:00 2022] PUT /identity/v3/projects/e1166ac77e8e403aa5f81e452bc039ac/users/5ca8be8093b547ee953ea646f90fdcae/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:00.645493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.647570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:00.666696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:00.710408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:00.715344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122af32-7b42-4a19-9621-ce04071e7e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.717045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122af32-7b42-4a19-9621-ce04071e7e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.717350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122af32-7b42-4a19-9621-ce04071e7e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.717629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122af32-7b42-4a19-9621-ce04071e7e96 admin admin] PATH_INFO: `/v3/projects/c4a9a9a51d7c4f7189bb934897f2c21a/users/b2306856ff3a4ad7addd8f8a6c0be45d/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.723843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90d12a-07b5-4ea5-a1fa-554c9d4cf931 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.728161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686a04e-9cb6-47bb-b513-e092f14db495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.728762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686a04e-9cb6-47bb-b513-e092f14db495 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.729107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686a04e-9cb6-47bb-b513-e092f14db495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.729444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686a04e-9cb6-47bb-b513-e092f14db495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.736362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122af32-7b42-4a19-9621-ce04071e7e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a9a9a51d7c4f7189bb934897f2c21a, user_id=b2306856ff3a4ad7addd8f8a6c0be45d, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.740556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122af32-7b42-4a19-9621-ce04071e7e96 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.740880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686a04e-9cb6-47bb-b513-e092f14db495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.742461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686a04e-9cb6-47bb-b513-e092f14db495 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.765184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 805/1606] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:00 2022] PUT /identity/v3/projects/c4a9a9a51d7c4f7189bb934897f2c21a/users/b2306856ff3a4ad7addd8f8a6c0be45d/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:00.765184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122af32-7b42-4a19-9621-ce04071e7e96 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.773724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 802/1607] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:00.776470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614636-317e-4027-afd4-f20be5d2a66b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.776769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614636-317e-4027-afd4-f20be5d2a66b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.780949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61614636-317e-4027-afd4-f20be5d2a66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.822545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-61614636-317e-4027-afd4-f20be5d2a66b None None] MFA Rules not processed for user `4234cd5ae2484fcc85c1e73b09af992b`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:00.824498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-61614636-317e-4027-afd4-f20be5d2a66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:00.867807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122af32-7b42-4a19-9621-ce04071e7e96 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.874132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b5ca6-ca23-4658-89be-e6f72676fd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:00.875184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5ca6-ca23-4658-89be-e6f72676fd82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.875607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5ca6-ca23-4658-89be-e6f72676fd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.875901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b5ca6-ca23-4658-89be-e6f72676fd82 admin admin] PATH_INFO: `/v3/projects/38cf167c2fc74ac09d6f6d6f81829cda/users/8a45cafadb824e08bd9e86fd0d5c4c71/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.898625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b5ca6-ca23-4658-89be-e6f72676fd82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38cf167c2fc74ac09d6f6d6f81829cda, user_id=8a45cafadb824e08bd9e86fd0d5c4c71, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:00.902291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b5ca6-ca23-4658-89be-e6f72676fd82 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:00.919659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 806/1608] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:00 2022] PUT /identity/v3/projects/38cf167c2fc74ac09d6f6d6f81829cda/users/8a45cafadb824e08bd9e86fd0d5c4c71/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:00.921000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95b22-a214-47ee-b07c-9c95df242e5e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:00.921378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95b22-a214-47ee-b07c-9c95df242e5e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:00.921739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e95b22-a214-47ee-b07c-9c95df242e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:00.929867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 803/1609] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:00.932751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61614636-317e-4027-afd4-f20be5d2a66b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:00.949188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61614636-317e-4027-afd4-f20be5d2a66b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:00.983815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-08e95b22-a214-47ee-b07c-9c95df242e5e None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:00.987271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-08e95b22-a214-47ee-b07c-9c95df242e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:01.063666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79edc20-648e-433a-8b7e-3bcaa950d216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.068333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79edc20-648e-433a-8b7e-3bcaa950d216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.068643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79edc20-648e-433a-8b7e-3bcaa950d216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.068897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79edc20-648e-433a-8b7e-3bcaa950d216 admin admin] PATH_INFO: `/v3/projects/e6a24dfb85b04adfa565f06fde95d6b6/users/3aab24bab8d74b7aa5a20e7b2b89b5d7/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.102578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79edc20-648e-433a-8b7e-3bcaa950d216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a24dfb85b04adfa565f06fde95d6b6, user_id=3aab24bab8d74b7aa5a20e7b2b89b5d7, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.108800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79edc20-648e-433a-8b7e-3bcaa950d216 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.127088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 807/1610] 10.222.0.71 () {58 vars in 1059 bytes} [Mon Jul 18 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:01.129381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e360f5d-e758-43dd-bdf6-a8d1dd2977cf None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.129844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e360f5d-e758-43dd-bdf6-a8d1dd2977cf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.130954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e360f5d-e758-43dd-bdf6-a8d1dd2977cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.145478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 804/1611] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:00 2022] PUT /identity/v3/projects/e6a24dfb85b04adfa565f06fde95d6b6/users/3aab24bab8d74b7aa5a20e7b2b89b5d7/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:01.145478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db213ca-0a8d-4121-96a9-2f17fc5c9239 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.145478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db213ca-0a8d-4121-96a9-2f17fc5c9239 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.146672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db213ca-0a8d-4121-96a9-2f17fc5c9239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.154103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8e360f5d-e758-43dd-bdf6-a8d1dd2977cf None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:01.155586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8e360f5d-e758-43dd-bdf6-a8d1dd2977cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:01.167765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:01.205611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7db213ca-0a8d-4121-96a9-2f17fc5c9239 None None] MFA Rules not processed for user `b2306856ff3a4ad7addd8f8a6c0be45d`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:01.207439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7db213ca-0a8d-4121-96a9-2f17fc5c9239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:01.276407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 808/1612] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:01.278185 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4bbe-deff-462b-9463-1e83bf09369e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.278467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4bbe-deff-462b-9463-1e83bf09369e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.278885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a4bbe-deff-462b-9463-1e83bf09369e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.307226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e50a4bbe-deff-462b-9463-1e83bf09369e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:01.308553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e50a4bbe-deff-462b-9463-1e83bf09369e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:01.321963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 809/1613] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:01.323409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4bbe-deff-462b-9463-1e83bf09369e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.328452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:01.340272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a4bbe-deff-462b-9463-1e83bf09369e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.348443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5ac74-84e2-4771-92e7-6db27cec0610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.349289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5ac74-84e2-4771-92e7-6db27cec0610 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.349573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5ac74-84e2-4771-92e7-6db27cec0610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.349943 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5ac74-84e2-4771-92e7-6db27cec0610 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.350846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5ac74-84e2-4771-92e7-6db27cec0610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.353484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5ac74-84e2-4771-92e7-6db27cec0610 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.371407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 805/1614] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:01.371407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db213ca-0a8d-4121-96a9-2f17fc5c9239 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.400450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 810/1615] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:01 2022] POST /identity/v3/projects => generated 379 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:01.405055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227583-11e7-43f3-8380-93a02cdf0d20 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.405408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227583-11e7-43f3-8380-93a02cdf0d20 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.407575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3227583-11e7-43f3-8380-93a02cdf0d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.444735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b3227583-11e7-43f3-8380-93a02cdf0d20 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:01.446113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b3227583-11e7-43f3-8380-93a02cdf0d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:01.538907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 811/1616] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:01.539972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227583-11e7-43f3-8380-93a02cdf0d20 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.557415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db213ca-0a8d-4121-96a9-2f17fc5c9239 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.609080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3227583-11e7-43f3-8380-93a02cdf0d20 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.614911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9276fe-fdc2-42d9-9046-c615caf6c240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.615889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9276fe-fdc2-42d9-9046-c615caf6c240 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.616320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9276fe-fdc2-42d9-9046-c615caf6c240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.616903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9276fe-fdc2-42d9-9046-c615caf6c240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.633767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9276fe-fdc2-42d9-9046-c615caf6c240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.640716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9276fe-fdc2-42d9-9046-c615caf6c240 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.650017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df327b49-bc4e-43c7-825b-9e0004b6ac72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.650806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327b49-bc4e-43c7-825b-9e0004b6ac72 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.651084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327b49-bc4e-43c7-825b-9e0004b6ac72 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.651411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df327b49-bc4e-43c7-825b-9e0004b6ac72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.657982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 812/1617] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:01.659211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9276fe-fdc2-42d9-9046-c615caf6c240 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.673771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9276fe-fdc2-42d9-9046-c615caf6c240 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.679710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051ccb2-9f15-47f5-be39-29c6adbd119d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.681156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ccb2-9f15-47f5-be39-29c6adbd119d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.681761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ccb2-9f15-47f5-be39-29c6adbd119d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.682192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051ccb2-9f15-47f5-be39-29c6adbd119d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.697099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051ccb2-9f15-47f5-be39-29c6adbd119d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.698809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051ccb2-9f15-47f5-be39-29c6adbd119d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.715292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 813/1618] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:01.717124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051ccb2-9f15-47f5-be39-29c6adbd119d service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.732775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051ccb2-9f15-47f5-be39-29c6adbd119d service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.736673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327b49-bc4e-43c7-825b-9e0004b6ac72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.739271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df327b49-bc4e-43c7-825b-9e0004b6ac72 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.744438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ec8e6-f264-44cb-ae87-196ee28c2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.745129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ec8e6-f264-44cb-ae87-196ee28c2e7d admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.745407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ec8e6-f264-44cb-ae87-196ee28c2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.745997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ec8e6-f264-44cb-ae87-196ee28c2e7d admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.747282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ec8e6-f264-44cb-ae87-196ee28c2e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.750847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ec8e6-f264-44cb-ae87-196ee28c2e7d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.757310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 806/1619] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:01.762339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6b0c-3c45-4cf5-957a-8aabd6f99ffe None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.762843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6b0c-3c45-4cf5-957a-8aabd6f99ffe None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.763283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e6b0c-3c45-4cf5-957a-8aabd6f99ffe None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.765486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 807/1620] 10.222.0.71 () {54 vars in 962 bytes} [Mon Jul 18 14:19:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:01.768969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e6b0c-3c45-4cf5-957a-8aabd6f99ffe None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.784667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e6b0c-3c45-4cf5-957a-8aabd6f99ffe None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.789239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:01.795126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60233663-184c-4781-bd75-236e310b2202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.795806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60233663-184c-4781-bd75-236e310b2202 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.796211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60233663-184c-4781-bd75-236e310b2202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.796516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60233663-184c-4781-bd75-236e310b2202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.804670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 814/1621] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:01 2022] POST /identity/v3/users => generated 296 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:01.806332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6226bc4-516d-41e5-a543-588148108280 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.807056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6226bc4-516d-41e5-a543-588148108280 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.807436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6226bc4-516d-41e5-a543-588148108280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.809360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60233663-184c-4781-bd75-236e310b2202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.810858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60233663-184c-4781-bd75-236e310b2202 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.822337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 808/1622] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:01.823182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60233663-184c-4781-bd75-236e310b2202 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.829397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a6226bc4-516d-41e5-a543-588148108280 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:01.831220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a6226bc4-516d-41e5-a543-588148108280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:01.836826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60233663-184c-4781-bd75-236e310b2202 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.842998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 815/1623] 10.222.0.71 () {58 vars in 1059 bytes} [Mon Jul 18 14:19:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:01.850314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2c1a2-d663-4b4c-8282-d9de2282d2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.851308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2c1a2-d663-4b4c-8282-d9de2282d2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.851595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2c1a2-d663-4b4c-8282-d9de2282d2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.851905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2c1a2-d663-4b4c-8282-d9de2282d2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.852259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6226bc4-516d-41e5-a543-588148108280 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.852624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f2c1a2-d663-4b4c-8282-d9de2282d2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.854409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f2c1a2-d663-4b4c-8282-d9de2282d2d9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.860701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:01.869313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 809/1624] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:01.874096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2c1a2-d663-4b4c-8282-d9de2282d2d9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:01.883112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6226bc4-516d-41e5-a543-588148108280 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.890091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2c1a2-d663-4b4c-8282-d9de2282d2d9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:01.893839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459dd65-2ed3-4c3d-8fd5-368b46c67dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.895375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459dd65-2ed3-4c3d-8fd5-368b46c67dad service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.896179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459dd65-2ed3-4c3d-8fd5-368b46c67dad service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.896909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459dd65-2ed3-4c3d-8fd5-368b46c67dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.905920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1468d92-70ed-475c-8381-fe0b081b24d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:01.910830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468d92-70ed-475c-8381-fe0b081b24d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:01.911585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468d92-70ed-475c-8381-fe0b081b24d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:01.912067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1468d92-70ed-475c-8381-fe0b081b24d3 admin admin] PATH_INFO: `/v3/projects/0b76e7d662a340a29dd6667fcf933acd/users/ae01abf11b6c49ac8cbd7e50e432e8a6/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:01.946594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1468d92-70ed-475c-8381-fe0b081b24d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b76e7d662a340a29dd6667fcf933acd, user_id=ae01abf11b6c49ac8cbd7e50e432e8a6, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:01.951665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1468d92-70ed-475c-8381-fe0b081b24d3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:01.986418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 810/1625] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:01 2022] PUT /identity/v3/projects/0b76e7d662a340a29dd6667fcf933acd/users/ae01abf11b6c49ac8cbd7e50e432e8a6/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:01.990517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1468d92-70ed-475c-8381-fe0b081b24d3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:02.057148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459dd65-2ed3-4c3d-8fd5-368b46c67dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:02.065142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459dd65-2ed3-4c3d-8fd5-368b46c67dad service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:02.120556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 816/1626] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:19:02.157328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1468d92-70ed-475c-8381-fe0b081b24d3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:02.182801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8e6b0-2112-47ae-97db-9b805586d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:02.183527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8e6b0-2112-47ae-97db-9b805586d917 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.185181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8e6b0-2112-47ae-97db-9b805586d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.185471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8e6b0-2112-47ae-97db-9b805586d917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.186359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8e6b0-2112-47ae-97db-9b805586d917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:02.194807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8e6b0-2112-47ae-97db-9b805586d917 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:02.233037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 811/1627] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:02.234343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459dd65-2ed3-4c3d-8fd5-368b46c67dad service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:02.258162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459dd65-2ed3-4c3d-8fd5-368b46c67dad service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:02.290525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a1ee1-e40d-4f57-9dc4-fac2d74511f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:02.296809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a1ee1-e40d-4f57-9dc4-fac2d74511f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.296809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a1ee1-e40d-4f57-9dc4-fac2d74511f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.296809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a1ee1-e40d-4f57-9dc4-fac2d74511f7 admin admin] PATH_INFO: `/v3/projects/0b76e7d662a340a29dd6667fcf933acd/users/ae01abf11b6c49ac8cbd7e50e432e8a6/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.304461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a1ee1-e40d-4f57-9dc4-fac2d74511f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b76e7d662a340a29dd6667fcf933acd, user_id=ae01abf11b6c49ac8cbd7e50e432e8a6, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:02.314394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a1ee1-e40d-4f57-9dc4-fac2d74511f7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:02.323752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48a1b3-4e1d-43cf-aed9-56eb7212484e None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.324340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48a1b3-4e1d-43cf-aed9-56eb7212484e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.324783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48a1b3-4e1d-43cf-aed9-56eb7212484e None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.326673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 812/1628] 10.222.0.71 () {54 vars in 912 bytes} [Mon Jul 18 14:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:02.333232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ba960-a71b-45d6-8c63-0836aa317f70 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.334014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ba960-a71b-45d6-8c63-0836aa317f70 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.334584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ba960-a71b-45d6-8c63-0836aa317f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.356909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 817/1629] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:02 2022] PUT /identity/v3/projects/0b76e7d662a340a29dd6667fcf933acd/users/ae01abf11b6c49ac8cbd7e50e432e8a6/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:02.367103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-695ba960-a71b-45d6-8c63-0836aa317f70 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:02.368854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-695ba960-a71b-45d6-8c63-0836aa317f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:02.369762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5069327-38b0-4a70-bade-dde063ac13cc None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.370112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5069327-38b0-4a70-bade-dde063ac13cc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.370399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5069327-38b0-4a70-bade-dde063ac13cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.524567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c5069327-38b0-4a70-bade-dde063ac13cc None None] MFA Rules not processed for user `ae01abf11b6c49ac8cbd7e50e432e8a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:02.524567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c5069327-38b0-4a70-bade-dde063ac13cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:02.528184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 813/1630] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:19:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:02.528184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ba960-a71b-45d6-8c63-0836aa317f70 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:02.700266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 818/1631] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:02.713515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5069327-38b0-4a70-bade-dde063ac13cc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:02.713857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ba960-a71b-45d6-8c63-0836aa317f70 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:02.747426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5069327-38b0-4a70-bade-dde063ac13cc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:02.752132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b799a-15eb-460b-b32a-e11df0848e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:02.752132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b799a-15eb-460b-b32a-e11df0848e8c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.753422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b799a-15eb-460b-b32a-e11df0848e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.753895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b799a-15eb-460b-b32a-e11df0848e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.762613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6542d-dbe2-4b16-ae66-7377a89fff4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:02.763373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6542d-dbe2-4b16-ae66-7377a89fff4d service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.763646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6542d-dbe2-4b16-ae66-7377a89fff4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.763922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6542d-dbe2-4b16-ae66-7377a89fff4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.780380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6542d-dbe2-4b16-ae66-7377a89fff4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:02.782192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6542d-dbe2-4b16-ae66-7377a89fff4d service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:02.798543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 819/1632] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:02.801334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3f9f2-c07d-40fb-a301-e651f4959cbc None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.802062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b799a-15eb-460b-b32a-e11df0848e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:02.802221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3f9f2-c07d-40fb-a301-e651f4959cbc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.802512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3f9f2-c07d-40fb-a301-e651f4959cbc None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.810410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 820/1633] 10.222.0.71 () {54 vars in 912 bytes} [Mon Jul 18 14:19:02 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:02.811016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b799a-15eb-460b-b32a-e11df0848e8c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:02.811941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ac056-9b91-47d2-8bc6-6b0d423e7212 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.812389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ac056-9b91-47d2-8bc6-6b0d423e7212 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.812661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ac056-9b91-47d2-8bc6-6b0d423e7212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.838954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f43ac056-9b91-47d2-8bc6-6b0d423e7212 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:02.841747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f43ac056-9b91-47d2-8bc6-6b0d423e7212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:02.849362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 814/1634] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:02.856029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 821/1635] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:19:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:02.937946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b799a-15eb-460b-b32a-e11df0848e8c service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:02.958711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b799a-15eb-460b-b32a-e11df0848e8c service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:02.976242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fffabb-fe50-4330-92c6-f5c3ee29bd78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:02.976925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fffabb-fe50-4330-92c6-f5c3ee29bd78 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:02.977274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fffabb-fe50-4330-92c6-f5c3ee29bd78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:02.977555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fffabb-fe50-4330-92c6-f5c3ee29bd78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:02.998583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:03.009131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fffabb-fe50-4330-92c6-f5c3ee29bd78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:03.011214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fffabb-fe50-4330-92c6-f5c3ee29bd78 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:03.067797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 815/1636] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:04.052549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ac056-9b91-47d2-8bc6-6b0d423e7212 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:04.063035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ac056-9b91-47d2-8bc6-6b0d423e7212 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:04.075996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc0d90-3ee1-47ef-831b-f68c80fc719b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:04.076689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0d90-3ee1-47ef-831b-f68c80fc719b service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:04.076973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0d90-3ee1-47ef-831b-f68c80fc719b service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:04.077251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc0d90-3ee1-47ef-831b-f68c80fc719b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:04.200036 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:04.238121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc0d90-3ee1-47ef-831b-f68c80fc719b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:04.239974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc0d90-3ee1-47ef-831b-f68c80fc719b service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:04.284160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 822/1637] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:06.505065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fffabb-fe50-4330-92c6-f5c3ee29bd78 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:06.692697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fffabb-fe50-4330-92c6-f5c3ee29bd78 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:06.714040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:06.715204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:06.715616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:06.716049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] PATH_INFO: `/v3/users/b2306856ff3a4ad7addd8f8a6c0be45d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:06.718014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2306856ff3a4ad7addd8f8a6c0be45d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:06.729320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:06.843701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2306856ff3a4ad7addd8f8a6c0be45d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:06.860317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2306856ff3a4ad7addd8f8a6c0be45d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:06.905151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2306856ff3a4ad7addd8f8a6c0be45d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:06.907055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2306856ff3a4ad7addd8f8a6c0be45d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:06.936101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2306856ff3a4ad7addd8f8a6c0be45d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:06.971305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 816/1638] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:06 2022] DELETE /identity/v3/users/b2306856ff3a4ad7addd8f8a6c0be45d => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:06.976270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda2c76-32c7-4465-b176-4a1a8e88dd67 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:06.976846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda2c76-32c7-4465-b176-4a1a8e88dd67 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:06.976846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda2c76-32c7-4465-b176-4a1a8e88dd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:07.003255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4eda2c76-32c7-4465-b176-4a1a8e88dd67 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:07.004870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4eda2c76-32c7-4465-b176-4a1a8e88dd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:07.096186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 823/1639] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:07.155762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:07.158385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:07.250628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13c039-236d-484e-b977-e55528f87bb5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:07.255509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175074b-52cb-4b82-b527-a94a5157186f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:07.256507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175074b-52cb-4b82-b527-a94a5157186f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:07.257006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175074b-52cb-4b82-b527-a94a5157186f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:07.257485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175074b-52cb-4b82-b527-a94a5157186f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:07.272364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175074b-52cb-4b82-b527-a94a5157186f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:07.275319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175074b-52cb-4b82-b527-a94a5157186f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:07.288446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 817/1640] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:08.655591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda2c76-32c7-4465-b176-4a1a8e88dd67 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:08.657815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4eda2c76-32c7-4465-b176-4a1a8e88dd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:08.678646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda2c76-32c7-4465-b176-4a1a8e88dd67 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:08.698299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:08.699791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:08.700343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:08.700823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] PATH_INFO: `/v3/projects/c4a9a9a51d7c4f7189bb934897f2c21a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:08.703977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a9a9a51d7c4f7189bb934897f2c21a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:08.706425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:08.828369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a9a9a51d7c4f7189bb934897f2c21a', 'request_id': 'req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:08.854840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a9a9a51d7c4f7189bb934897f2c21a', 'request_id': 'req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:08.872430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a9a9a51d7c4f7189bb934897f2c21a', 'request_id': 'req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:08.919809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] The token cache is being invalidate because project c4a9a9a51d7c4f7189bb934897f2c21a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:08.922738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:08.928029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e6419220-b17d-4613-a0ca-6ec0e3e9aeea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:08.931097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 824/1641] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:08 2022] DELETE /identity/v3/projects/c4a9a9a51d7c4f7189bb934897f2c21a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:08.936176 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811867c2-88a9-4f0c-aafa-9e6641b8eba5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:08.937196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811867c2-88a9-4f0c-aafa-9e6641b8eba5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:08.937728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811867c2-88a9-4f0c-aafa-9e6641b8eba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:08.988674 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-811867c2-88a9-4f0c-aafa-9e6641b8eba5 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:08.991669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-811867c2-88a9-4f0c-aafa-9e6641b8eba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:09.024776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:09.249466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 818/1642] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:09.258860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd302fc-1f03-4880-ae9b-4314d52caefc None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:09.259438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd302fc-1f03-4880-ae9b-4314d52caefc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:09.259923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd302fc-1f03-4880-ae9b-4314d52caefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:09.306875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2fd302fc-1f03-4880-ae9b-4314d52caefc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:09.315558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2fd302fc-1f03-4880-ae9b-4314d52caefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:09.337870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 825/1643] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:09.342379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811867c2-88a9-4f0c-aafa-9e6641b8eba5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:09.355038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811867c2-88a9-4f0c-aafa-9e6641b8eba5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:09.378023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:09.378911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:09.379330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:09.379715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:09.380579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:09.380999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:09.386251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:09.419497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 819/1644] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:09.424202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd302fc-1f03-4880-ae9b-4314d52caefc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:09.456502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd302fc-1f03-4880-ae9b-4314d52caefc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:09.480241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bd285-2be6-48dc-b824-9e3e5e210523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:09.480927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bd285-2be6-48dc-b824-9e3e5e210523 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:09.481207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bd285-2be6-48dc-b824-9e3e5e210523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:09.481508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bd285-2be6-48dc-b824-9e3e5e210523 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:09.482495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bd285-2be6-48dc-b824-9e3e5e210523 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:09.484524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bd285-2be6-48dc-b824-9e3e5e210523 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:09.604215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 826/1645] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:09 2022] POST /identity/v3/projects => generated 403 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:09.609742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:09.623778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1c1a0-61e6-455d-9964-9851d0a51d92 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:09.816172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec55e-3da1-43c1-b5a1-684b1afc62ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:09.816889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec55e-3da1-43c1-b5a1-684b1afc62ba admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:09.817220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec55e-3da1-43c1-b5a1-684b1afc62ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:09.817588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec55e-3da1-43c1-b5a1-684b1afc62ba admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:09.823181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ec55e-3da1-43c1-b5a1-684b1afc62ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:09.825588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ec55e-3da1-43c1-b5a1-684b1afc62ba admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:09.917321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 820/1646] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:09 2022] POST /identity/v3/users => generated 302 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:09.918766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bd285-2be6-48dc-b824-9e3e5e210523 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:09.943828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bd285-2be6-48dc-b824-9e3e5e210523 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:09.971970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8773dc9-e199-429b-baea-605469be1bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:09.978458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8773dc9-e199-429b-baea-605469be1bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:09.979080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8773dc9-e199-429b-baea-605469be1bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:09.979887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8773dc9-e199-429b-baea-605469be1bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:09.981073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8773dc9-e199-429b-baea-605469be1bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:09.982784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8773dc9-e199-429b-baea-605469be1bcd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:10.022048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 827/1647] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:10.026162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec55e-3da1-43c1-b5a1-684b1afc62ba admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:10.046964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ec55e-3da1-43c1-b5a1-684b1afc62ba admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:10.053472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c4509-e053-46c3-8683-d1bd97450029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:10.054694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c4509-e053-46c3-8683-d1bd97450029 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:10.055193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c4509-e053-46c3-8683-d1bd97450029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:10.055836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c4509-e053-46c3-8683-d1bd97450029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:10.056873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c4509-e053-46c3-8683-d1bd97450029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:10.058941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c4509-e053-46c3-8683-d1bd97450029 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:10.076048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 821/1648] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:10.080070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8773dc9-e199-429b-baea-605469be1bcd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:10.107599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8773dc9-e199-429b-baea-605469be1bcd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:10.119682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7828d-06d7-44b5-913a-59387577824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:10.122397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7828d-06d7-44b5-913a-59387577824f admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:10.123856 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7828d-06d7-44b5-913a-59387577824f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:10.124510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7828d-06d7-44b5-913a-59387577824f admin admin] PATH_INFO: `/v3/projects/81dd9726574f4b5290df49e298a3609b/users/790683f20c46494ab3e76abe5be4eaf1/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:10.164136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7828d-06d7-44b5-913a-59387577824f admin admin] RBAC: Authorizing `identity:create_grant(project_id=81dd9726574f4b5290df49e298a3609b, user_id=790683f20c46494ab3e76abe5be4eaf1, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:10.167292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7828d-06d7-44b5-913a-59387577824f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:10.197923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:10.242271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 828/1649] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:10 2022] PUT /identity/v3/projects/81dd9726574f4b5290df49e298a3609b/users/790683f20c46494ab3e76abe5be4eaf1/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:10.244535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc4839-1fa9-478f-b265-3332c323fe1e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:10.245479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc4839-1fa9-478f-b265-3332c323fe1e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:10.245479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc4839-1fa9-478f-b265-3332c323fe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:10.355403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-19cc4839-1fa9-478f-b265-3332c323fe1e None None] MFA Rules not processed for user `790683f20c46494ab3e76abe5be4eaf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:10.358760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-19cc4839-1fa9-478f-b265-3332c323fe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:10.520515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 822/1650] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:10.524852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7828d-06d7-44b5-913a-59387577824f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:10.762467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7828d-06d7-44b5-913a-59387577824f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:10.783629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f2b4-8e5c-4707-bba8-a8fec2335fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:10.784152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f2b4-8e5c-4707-bba8-a8fec2335fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:10.784557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f2b4-8e5c-4707-bba8-a8fec2335fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:10.785079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804f2b4-8e5c-4707-bba8-a8fec2335fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:10.786026 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804f2b4-8e5c-4707-bba8-a8fec2335fa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:10.788342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804f2b4-8e5c-4707-bba8-a8fec2335fa6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:10.853247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 829/1651] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:10 2022] POST /identity/v3/projects => generated 405 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:10.857191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc4839-1fa9-478f-b265-3332c323fe1e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:10.887024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc4839-1fa9-478f-b265-3332c323fe1e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:10.917468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:11.155445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6a803-17c4-4a4c-90a0-f4b20ef75d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:11.162688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6a803-17c4-4a4c-90a0-f4b20ef75d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:11.162688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6a803-17c4-4a4c-90a0-f4b20ef75d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:11.162956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6a803-17c4-4a4c-90a0-f4b20ef75d9e admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:11.164814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6a803-17c4-4a4c-90a0-f4b20ef75d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:11.169847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6a803-17c4-4a4c-90a0-f4b20ef75d9e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:11.267002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 823/1652] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:10 2022] POST /identity/v3/users => generated 309 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:11.268299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f2b4-8e5c-4707-bba8-a8fec2335fa6 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:11.282277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804f2b4-8e5c-4707-bba8-a8fec2335fa6 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:11.316155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624a361-48dd-4d6c-a071-c523453f6254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:11.317242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624a361-48dd-4d6c-a071-c523453f6254 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:11.317754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624a361-48dd-4d6c-a071-c523453f6254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:11.318297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8624a361-48dd-4d6c-a071-c523453f6254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:11.319262 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624a361-48dd-4d6c-a071-c523453f6254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:11.321409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8624a361-48dd-4d6c-a071-c523453f6254 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:11.372701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 830/1653] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:11.377875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6a803-17c4-4a4c-90a0-f4b20ef75d9e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:11.391099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6a803-17c4-4a4c-90a0-f4b20ef75d9e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:11.421436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a2055-7b2a-4a69-896e-4696dd98cd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:11.423252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a2055-7b2a-4a69-896e-4696dd98cd72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:11.424458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a2055-7b2a-4a69-896e-4696dd98cd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:11.429910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a2055-7b2a-4a69-896e-4696dd98cd72 admin admin] PATH_INFO: `/v3/projects/0e3cc9ae9876444ab9b3207e534afc4c/users/66b76333cd9f49b9be19bbd3cfa0c042/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:11.472941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a2055-7b2a-4a69-896e-4696dd98cd72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3cc9ae9876444ab9b3207e534afc4c, user_id=66b76333cd9f49b9be19bbd3cfa0c042, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:11.484428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a2055-7b2a-4a69-896e-4696dd98cd72 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:11.573910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 824/1654] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:11 2022] PUT /identity/v3/projects/0e3cc9ae9876444ab9b3207e534afc4c/users/66b76333cd9f49b9be19bbd3cfa0c042/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:11.577624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624a361-48dd-4d6c-a071-c523453f6254 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:11.589578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8624a361-48dd-4d6c-a071-c523453f6254 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:11.766703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3140d-a566-454d-ba86-71c3ca31448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:11.767756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3140d-a566-454d-ba86-71c3ca31448d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:11.768205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3140d-a566-454d-ba86-71c3ca31448d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:11.768659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3140d-a566-454d-ba86-71c3ca31448d admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:11.770126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3140d-a566-454d-ba86-71c3ca31448d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:11.782748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3140d-a566-454d-ba86-71c3ca31448d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:11.817528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 831/1655] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:11.822437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a2055-7b2a-4a69-896e-4696dd98cd72 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:11.844522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a2055-7b2a-4a69-896e-4696dd98cd72 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:11.859570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:11.862150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:11.862438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:11.864102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] PATH_INFO: `/v3/projects/0e3cc9ae9876444ab9b3207e534afc4c/users/66b76333cd9f49b9be19bbd3cfa0c042/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:11.874381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3cc9ae9876444ab9b3207e534afc4c, user_id=66b76333cd9f49b9be19bbd3cfa0c042, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:11.880420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:11.930097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 825/1656] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:11 2022] PUT /identity/v3/projects/0e3cc9ae9876444ab9b3207e534afc4c/users/66b76333cd9f49b9be19bbd3cfa0c042/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:11.943146 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9287c4-2e47-42e0-ab37-5522c1691fdf None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:11.943550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9287c4-2e47-42e0-ab37-5522c1691fdf None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:11.943853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9287c4-2e47-42e0-ab37-5522c1691fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.013117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0b9287c4-2e47-42e0-ab37-5522c1691fdf None None] MFA Rules not processed for user `66b76333cd9f49b9be19bbd3cfa0c042`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:12.014866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0b9287c4-2e47-42e0-ab37-5522c1691fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:12.230697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:12.256264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 832/1657] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:12.272553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:12.274376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:12.400986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd077a44-d493-4b3d-880d-497b4c8c294a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:12.413135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da9cd5-5914-4534-8d4a-caedfea0b139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:12.414321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da9cd5-5914-4534-8d4a-caedfea0b139 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:12.415885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da9cd5-5914-4534-8d4a-caedfea0b139 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:12.416441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da9cd5-5914-4534-8d4a-caedfea0b139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:12.441264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da9cd5-5914-4534-8d4a-caedfea0b139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:12.445687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da9cd5-5914-4534-8d4a-caedfea0b139 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:12.473189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 826/1658] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:12 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:15.257893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9287c4-2e47-42e0-ab37-5522c1691fdf None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:15.476589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9287c4-2e47-42e0-ab37-5522c1691fdf None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:15.492589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:15.495156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.495460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.497103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] PATH_INFO: `/v3/users/790683f20c46494ab3e76abe5be4eaf1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:15.502230 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=790683f20c46494ab3e76abe5be4eaf1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:15.506532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: WARNING py.warnings [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:19:15.506532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: warnings.warn(msg) Jul 18 14:19:15.506532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: Jul 18 14:19:15.511862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:15.790469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790683f20c46494ab3e76abe5be4eaf1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.809738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790683f20c46494ab3e76abe5be4eaf1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.887505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790683f20c46494ab3e76abe5be4eaf1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.888824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790683f20c46494ab3e76abe5be4eaf1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.891503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790683f20c46494ab3e76abe5be4eaf1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:15.948384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 833/1659] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:15 2022] DELETE /identity/v3/users/790683f20c46494ab3e76abe5be4eaf1 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:15.949805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf98001-d82a-4c80-a9cc-59db44a196b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:15.950307 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf98001-d82a-4c80-a9cc-59db44a196b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:15.950417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf98001-d82a-4c80-a9cc-59db44a196b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.002943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-dbf98001-d82a-4c80-a9cc-59db44a196b0 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:16.004276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dbf98001-d82a-4c80-a9cc-59db44a196b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.206224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 827/1660] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:16.236150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.238901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.526155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87afb9b1-81d9-4870-94f2-ddad617ee7e5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.532819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec3248-4a8a-4358-bc73-d83c8b16807d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.537689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec3248-4a8a-4358-bc73-d83c8b16807d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.538340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec3248-4a8a-4358-bc73-d83c8b16807d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.538774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec3248-4a8a-4358-bc73-d83c8b16807d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:16.563691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec3248-4a8a-4358-bc73-d83c8b16807d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:16.570433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ec3248-4a8a-4358-bc73-d83c8b16807d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:16.606465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 834/1661] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:16.869677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf98001-d82a-4c80-a9cc-59db44a196b0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:16.872064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dbf98001-d82a-4c80-a9cc-59db44a196b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:16.932613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf98001-d82a-4c80-a9cc-59db44a196b0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:16.979709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:16.983721 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:16.984235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:16.984730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] PATH_INFO: `/v3/users/93f4adb167914e1b82542cbd87f3f214` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.000173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f4adb167914e1b82542cbd87f3f214)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:17.013485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:17.016877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:17.204886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f4adb167914e1b82542cbd87f3f214'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.227880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f4adb167914e1b82542cbd87f3f214'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.301417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f4adb167914e1b82542cbd87f3f214'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.303181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f4adb167914e1b82542cbd87f3f214'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.322693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-528f32e1-81af-4ded-9a5b-f8cef5bef5a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f4adb167914e1b82542cbd87f3f214'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:17.409296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 828/1662] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:16 2022] DELETE /identity/v3/users/93f4adb167914e1b82542cbd87f3f214 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:17.416592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c6298-0176-4048-8d49-872c06c4d996 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.417543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c6298-0176-4048-8d49-872c06c4d996 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.418254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c6298-0176-4048-8d49-872c06c4d996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.503496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-157c6298-0176-4048-8d49-872c06c4d996 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:17.516233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-157c6298-0176-4048-8d49-872c06c4d996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:17.561117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:17.568341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2d34e-5f94-4937-9bb6-4073270f979d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.568927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2d34e-5f94-4937-9bb6-4073270f979d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.569446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2d34e-5f94-4937-9bb6-4073270f979d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.721984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fda2d34e-5f94-4937-9bb6-4073270f979d None None] MFA Rules not processed for user `2e3c7eb6a5cb4515b6f48520535fea76`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:17.723546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fda2d34e-5f94-4937-9bb6-4073270f979d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:17.837731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 835/1663] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:17.839280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298cad-9f76-4b5a-af5b-ae4210a8bcb4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:17.839580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298cad-9f76-4b5a-af5b-ae4210a8bcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:17.839865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8298cad-9f76-4b5a-af5b-ae4210a8bcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:17.957123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b8298cad-9f76-4b5a-af5b-ae4210a8bcb4 None None] MFA Rules not processed for user `3aab24bab8d74b7aa5a20e7b2b89b5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:17.962789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b8298cad-9f76-4b5a-af5b-ae4210a8bcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:18.026207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 829/1664] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:18.031085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2d34e-5f94-4937-9bb6-4073270f979d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:18.054926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fda2d34e-5f94-4937-9bb6-4073270f979d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:18.296421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 836/1665] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:18.300087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e769b2-b75e-4fa6-a026-bc0f2094b722 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:18.303768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e769b2-b75e-4fa6-a026-bc0f2094b722 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:18.304119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e769b2-b75e-4fa6-a026-bc0f2094b722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:18.318876 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2d34e-5f94-4937-9bb6-4073270f979d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:18.345454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:18.348249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:18.348575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:18.349054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:18.357639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-44e769b2-b75e-4fa6-a026-bc0f2094b722 None None] MFA Rules not processed for user `2e3c7eb6a5cb4515b6f48520535fea76`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:18.359002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-44e769b2-b75e-4fa6-a026-bc0f2094b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:18.370084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 837/1666] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:18.371879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e769b2-b75e-4fa6-a026-bc0f2094b722 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:18.373741 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-44e769b2-b75e-4fa6-a026-bc0f2094b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:18.385998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:18.387834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:18.410098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 830/1667] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:18.413486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:18.422888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:18.457409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:18.630838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e769b2-b75e-4fa6-a026-bc0f2094b722 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:18.663997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:18.669722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:18.671411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:18.672183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:18.712431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:18.725088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:18.729718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe3a2b-8b21-4ba4-8985-c27d25e7fcd9 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:18.773809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccdb4b-7110-4cf4-91ae-f261ca9ba996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:18.777850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdb4b-7110-4cf4-91ae-f261ca9ba996 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:18.782788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 838/1668] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:18.784956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdb4b-7110-4cf4-91ae-f261ca9ba996 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:18.784956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccdb4b-7110-4cf4-91ae-f261ca9ba996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:18.816401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccdb4b-7110-4cf4-91ae-f261ca9ba996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:18.819891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adccdb4b-7110-4cf4-91ae-f261ca9ba996 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:18.862163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 831/1669] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:20.039830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:20.042786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:20.087368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f7bfb-8485-47d3-bf36-17ce84587d63 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:20.109165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:20.110842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:20.111363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:20.111870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] PATH_INFO: `/v3/projects/81dd9726574f4b5290df49e298a3609b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:20.116712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=81dd9726574f4b5290df49e298a3609b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:20.122743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:20.224713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:20.337273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81dd9726574f4b5290df49e298a3609b', 'request_id': 'req-0f87350b-a717-49c5-9575-809a3cf38cce'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.361512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81dd9726574f4b5290df49e298a3609b', 'request_id': 'req-0f87350b-a717-49c5-9575-809a3cf38cce'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.412640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81dd9726574f4b5290df49e298a3609b', 'request_id': 'req-0f87350b-a717-49c5-9575-809a3cf38cce'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.477919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] The token cache is being invalidate because project 81dd9726574f4b5290df49e298a3609b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:20.482461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.484788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.494012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 839/1670] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:20 2022] DELETE /identity/v3/projects/81dd9726574f4b5290df49e298a3609b => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:20.499512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccdb4b-7110-4cf4-91ae-f261ca9ba996 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:20.535860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccdb4b-7110-4cf4-91ae-f261ca9ba996 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:20.656785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:20.659673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:20.737039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:20.739679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:20.740275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:20.740700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] PATH_INFO: `/v3/users/66b76333cd9f49b9be19bbd3cfa0c042` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:20.742495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66b76333cd9f49b9be19bbd3cfa0c042)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:20.744755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:20.860167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66b76333cd9f49b9be19bbd3cfa0c042'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.877177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66b76333cd9f49b9be19bbd3cfa0c042'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.898850 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87350b-a717-49c5-9575-809a3cf38cce admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:20.913876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:20.915249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66b76333cd9f49b9be19bbd3cfa0c042'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.916267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66b76333cd9f49b9be19bbd3cfa0c042'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.916902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82fd9540-260f-4ac1-b0b5-5c4b4d030517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66b76333cd9f49b9be19bbd3cfa0c042'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:20.918019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:20.922610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:20.923773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] PATH_INFO: `/v3/users/4234cd5ae2484fcc85c1e73b09af992b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:20.927796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=4234cd5ae2484fcc85c1e73b09af992b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:20.934109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 832/1671] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:20 2022] DELETE /identity/v3/users/66b76333cd9f49b9be19bbd3cfa0c042 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:20.935833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3c19-343d-44ee-84ba-b03001f239fe None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:20.936392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3c19-343d-44ee-84ba-b03001f239fe None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:20.936779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e3c19-343d-44ee-84ba-b03001f239fe None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:20.938700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 833/1672] 10.222.0.71 () {54 vars in 912 bytes} [Mon Jul 18 14:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:20.939743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:20.949135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293e69b-d42d-4c8c-b9f8-4282f967d953 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:20.949796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293e69b-d42d-4c8c-b9f8-4282f967d953 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:20.950209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293e69b-d42d-4c8c-b9f8-4282f967d953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:21.022177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:21.046920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d293e69b-d42d-4c8c-b9f8-4282f967d953 None None] MFA Rules not processed for user `1f1fe084c3164e50bf5d904350925f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:21.052658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d293e69b-d42d-4c8c-b9f8-4282f967d953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:21.168661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4234cd5ae2484fcc85c1e73b09af992b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:21.194490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4234cd5ae2484fcc85c1e73b09af992b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:21.254806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4234cd5ae2484fcc85c1e73b09af992b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:21.257261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4234cd5ae2484fcc85c1e73b09af992b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:21.264461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4234cd5ae2484fcc85c1e73b09af992b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:21.301272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 834/1673] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:19:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:21.323481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 840/1674] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:20 2022] DELETE /identity/v3/users/4234cd5ae2484fcc85c1e73b09af992b => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:19:21.329487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73116d7-4ef0-4332-8918-34e6afccdac1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:21.330019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73116d7-4ef0-4332-8918-34e6afccdac1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:21.330505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73116d7-4ef0-4332-8918-34e6afccdac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:21.390623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b73116d7-4ef0-4332-8918-34e6afccdac1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:21.392810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b73116d7-4ef0-4332-8918-34e6afccdac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:21.554566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:21.562473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:21.598512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 835/1675] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:21.667547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ce6d-2800-4c2e-89e0-6c48cb31a731 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:21.668694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ce6d-2800-4c2e-89e0-6c48cb31a731 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:21.670394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ce6d-2800-4c2e-89e0-6c48cb31a731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:21.741848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-61c3ce6d-2800-4c2e-89e0-6c48cb31a731 None None] MFA Rules not processed for user `a0210f69010b4529822bcdac1bdf8b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:21.745558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-61c3ce6d-2800-4c2e-89e0-6c48cb31a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:21.754115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:21.871316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537536d6-32be-430f-b6e8-a37f6ff259ba admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:21.889512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b1f5b-f9cb-45f5-a8df-200a2f90683f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:21.890314 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b1f5b-f9cb-45f5-a8df-200a2f90683f service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:21.890659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b1f5b-f9cb-45f5-a8df-200a2f90683f service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:21.890967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b1f5b-f9cb-45f5-a8df-200a2f90683f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:21.932486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b1f5b-f9cb-45f5-a8df-200a2f90683f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:21.934179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b1f5b-f9cb-45f5-a8df-200a2f90683f service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:21.955957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 841/1676] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:21.961763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b1f5b-f9cb-45f5-a8df-200a2f90683f service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:22.013941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b1f5b-f9cb-45f5-a8df-200a2f90683f service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:22.038779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fa9742-bcc6-4db9-8b81-83a119cceed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:22.040618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9742-bcc6-4db9-8b81-83a119cceed6 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:22.041974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9742-bcc6-4db9-8b81-83a119cceed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:22.042742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fa9742-bcc6-4db9-8b81-83a119cceed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:22.052449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 836/1677] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:22.053496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ce6d-2800-4c2e-89e0-6c48cb31a731 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:22.063203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-61c3ce6d-2800-4c2e-89e0-6c48cb31a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:22.080728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa9742-bcc6-4db9-8b81-83a119cceed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:22.087259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fa9742-bcc6-4db9-8b81-83a119cceed6 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:22.106893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ce6d-2800-4c2e-89e0-6c48cb31a731 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:22.117961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 842/1678] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:22.119341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef5060-fb56-4c15-8bfa-90ef35ca01e1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:22.119641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef5060-fb56-4c15-8bfa-90ef35ca01e1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:22.119927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef5060-fb56-4c15-8bfa-90ef35ca01e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:22.126215 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:22.127153 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:22.127438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:22.127713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] PATH_INFO: `/v3/projects/c64ef5c9e4e74877afbe0b977348706e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:22.134560 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c64ef5c9e4e74877afbe0b977348706e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:22.136191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: WARNING py.warnings [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 18 14:19:22.136191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: warnings.warn(msg) Jul 18 14:19:22.136191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: Jul 18 14:19:22.136842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:22.182594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-06ef5060-fb56-4c15-8bfa-90ef35ca01e1 None None] MFA Rules not processed for user `8a45cafadb824e08bd9e86fd0d5c4c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:22.183716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-06ef5060-fb56-4c15-8bfa-90ef35ca01e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:22.268470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:22.326533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c64ef5c9e4e74877afbe0b977348706e', 'request_id': 'req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:22.341513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c64ef5c9e4e74877afbe0b977348706e', 'request_id': 'req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:22.376321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 843/1679] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:22.376321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef5060-fb56-4c15-8bfa-90ef35ca01e1 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:22.376321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-06ef5060-fb56-4c15-8bfa-90ef35ca01e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:22.376733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c64ef5c9e4e74877afbe0b977348706e', 'request_id': 'req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:22.402205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] The token cache is being invalidate because project c64ef5c9e4e74877afbe0b977348706e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:22.402744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:22.404024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-198219a0-e8e8-41b6-9f25-5e2c2ff9996d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:22.407090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 837/1680] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:22 2022] DELETE /identity/v3/projects/c64ef5c9e4e74877afbe0b977348706e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:22.409546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d733a82-0341-4ba8-af76-7b2b39748a63 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:22.411123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d733a82-0341-4ba8-af76-7b2b39748a63 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:22.412610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d733a82-0341-4ba8-af76-7b2b39748a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:22.444089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8d733a82-0341-4ba8-af76-7b2b39748a63 None None] MFA Rules not processed for user `8a45cafadb824e08bd9e86fd0d5c4c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:22.450566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8d733a82-0341-4ba8-af76-7b2b39748a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:22.561041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef5060-fb56-4c15-8bfa-90ef35ca01e1 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:22.609994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 838/1681] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:22.613381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc697-43f5-4439-8a0c-93674a5cb156 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:22.613733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc697-43f5-4439-8a0c-93674a5cb156 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:22.614771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdc697-43f5-4439-8a0c-93674a5cb156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:22.683664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d2cdc697-43f5-4439-8a0c-93674a5cb156 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:22.687652 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d2cdc697-43f5-4439-8a0c-93674a5cb156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:22.702316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:22.765569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:22.766416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:22.767320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:22.767573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:22.772696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:22.875815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 839/1682] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:22.885970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831703-2de6-4189-80d9-bc5fb2710ea7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:22.886401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831703-2de6-4189-80d9-bc5fb2710ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:22.886808 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831703-2de6-4189-80d9-bc5fb2710ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:22.927106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3d831703-2de6-4189-80d9-bc5fb2710ea7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:22.928778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3d831703-2de6-4189-80d9-bc5fb2710ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:22.951109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 840/1683] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:22.952564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d831703-2de6-4189-80d9-bc5fb2710ea7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:22.964982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:22.966734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:22.979862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d831703-2de6-4189-80d9-bc5fb2710ea7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:22.985046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 844/1684] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:22.996186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:22.999019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.000558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.000964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.004420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:23.007880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:23.012853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:23.045749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 841/1685] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:23.049013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:23.076545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba63d9-0bd1-4eca-9dbd-233c6e038aeb service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:23.092897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29ffa3-5e23-41dc-b0c9-384143fdc928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:23.093591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29ffa3-5e23-41dc-b0c9-384143fdc928 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.107578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:23.108005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29ffa3-5e23-41dc-b0c9-384143fdc928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.108462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29ffa3-5e23-41dc-b0c9-384143fdc928 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.109232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29ffa3-5e23-41dc-b0c9-384143fdc928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:23.110722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:23.111309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29ffa3-5e23-41dc-b0c9-384143fdc928 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:23.139099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd36c1f-4f53-4152-94f9-6427dce01d1d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:23.154130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c809fb-0d46-42f2-b468-cb64f1c53507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:23.156430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c809fb-0d46-42f2-b468-cb64f1c53507 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.156988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c809fb-0d46-42f2-b468-cb64f1c53507 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.157907 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c809fb-0d46-42f2-b468-cb64f1c53507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.160620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-36c809fb-0d46-42f2-b468-cb64f1c53507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:23.165257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 845/1686] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:23 2022] POST /identity/v3/projects => generated 363 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:23.170130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29ffa3-5e23-41dc-b0c9-384143fdc928 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:23.186540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c809fb-0d46-42f2-b468-cb64f1c53507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:23.196013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c809fb-0d46-42f2-b468-cb64f1c53507 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:23.227488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 842/1687] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:23.229547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4755829-7aef-4e33-a663-df28928cd22f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.231947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4755829-7aef-4e33-a663-df28928cd22f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.236461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4755829-7aef-4e33-a663-df28928cd22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.323054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f4755829-7aef-4e33-a663-df28928cd22f None None] MFA Rules not processed for user `5ca8be8093b547ee953ea646f90fdcae`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:23.324717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f4755829-7aef-4e33-a663-df28928cd22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:23.352987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29ffa3-5e23-41dc-b0c9-384143fdc928 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:23.388343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:23.389405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.389946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.390515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.396346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:23.399854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:23.440906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:23.460381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 846/1688] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:23 2022] POST /identity/v3/users => generated 282 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:23.461451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:23.466822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:23.523507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eeb912-0c18-4089-b5a3-3a655817b9c0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:23.542408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:23.543441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.543834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.544220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] PATH_INFO: `/v3/projects/0e3cc9ae9876444ab9b3207e534afc4c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.551934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3cc9ae9876444ab9b3207e534afc4c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:23.555683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:23.558143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 843/1689] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:23.559498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4755829-7aef-4e33-a663-df28928cd22f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:23.561529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f4755829-7aef-4e33-a663-df28928cd22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:23.589693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4755829-7aef-4e33-a663-df28928cd22f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:23.606318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:23.610422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.611508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.611804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] PATH_INFO: `/v3/projects/b2c6da087ce74a1d8ef8cbad1d1506ae` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.617770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c6da087ce74a1d8ef8cbad1d1506ae)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:23.622329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:23.694703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:23.703284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3cc9ae9876444ab9b3207e534afc4c', 'request_id': 'req-da1bea2c-da86-4170-a7a6-7e4a63f4a170'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.719116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3cc9ae9876444ab9b3207e534afc4c', 'request_id': 'req-da1bea2c-da86-4170-a7a6-7e4a63f4a170'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.746032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3cc9ae9876444ab9b3207e534afc4c', 'request_id': 'req-da1bea2c-da86-4170-a7a6-7e4a63f4a170'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.761502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c6da087ce74a1d8ef8cbad1d1506ae', 'request_id': 'req-b35211d0-3425-425b-b8e1-c2fc89be3f7e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.772095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] The token cache is being invalidate because project 0e3cc9ae9876444ab9b3207e534afc4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:23.772449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.773307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.779254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 847/1690] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:23 2022] DELETE /identity/v3/projects/0e3cc9ae9876444ab9b3207e534afc4c => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:23.779254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:23.779534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c6da087ce74a1d8ef8cbad1d1506ae', 'request_id': 'req-b35211d0-3425-425b-b8e1-c2fc89be3f7e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.783664 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:23.792700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c6da087ce74a1d8ef8cbad1d1506ae', 'request_id': 'req-b35211d0-3425-425b-b8e1-c2fc89be3f7e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.820413 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] The token cache is being invalidate because project b2c6da087ce74a1d8ef8cbad1d1506ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:23.820827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.821854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b35211d0-3425-425b-b8e1-c2fc89be3f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:23.824541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 844/1691] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:23 2022] DELETE /identity/v3/projects/b2c6da087ce74a1d8ef8cbad1d1506ae => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:23.828869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8671f-8d96-47f3-b7fa-7da395ef824c None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.833578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8671f-8d96-47f3-b7fa-7da395ef824c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.833955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8671f-8d96-47f3-b7fa-7da395ef824c None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.835730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 845/1692] 10.222.0.71 () {54 vars in 961 bytes} [Mon Jul 18 14:19:23 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:23.842747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f34e0-6945-44b1-af52-a48c2f23742d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:23.844774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f34e0-6945-44b1-af52-a48c2f23742d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:23.846447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f34e0-6945-44b1-af52-a48c2f23742d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:23.914141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-491f34e0-6945-44b1-af52-a48c2f23742d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:23.919203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-491f34e0-6945-44b1-af52-a48c2f23742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:23.923056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bea2c-da86-4170-a7a6-7e4a63f4a170 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:24.063679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 846/1693] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:24.063679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f34e0-6945-44b1-af52-a48c2f23742d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:24.063679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-491f34e0-6945-44b1-af52-a48c2f23742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:24.080613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0713f-4e6c-41a0-9c12-863eef21ffe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:24.081421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0713f-4e6c-41a0-9c12-863eef21ffe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.082180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0713f-4e6c-41a0-9c12-863eef21ffe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.082512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0713f-4e6c-41a0-9c12-863eef21ffe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.084918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0713f-4e6c-41a0-9c12-863eef21ffe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:24.087058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0713f-4e6c-41a0-9c12-863eef21ffe5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:24.088036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f34e0-6945-44b1-af52-a48c2f23742d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:24.104385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 848/1694] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:24.104555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:24.104676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce812d0-4e05-4a85-9493-cd2a21ce7146 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.104676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce812d0-4e05-4a85-9493-cd2a21ce7146 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.104804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.104804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.104937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce812d0-4e05-4a85-9493-cd2a21ce7146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.105031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] PATH_INFO: `/v3/users/ae01abf11b6c49ac8cbd7e50e432e8a6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.106134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae01abf11b6c49ac8cbd7e50e432e8a6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:24.108114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:24.132421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-dce812d0-4e05-4a85-9493-cd2a21ce7146 None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:24.133822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dce812d0-4e05-4a85-9493-cd2a21ce7146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:24.215094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:24.217508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae01abf11b6c49ac8cbd7e50e432e8a6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:24.231047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae01abf11b6c49ac8cbd7e50e432e8a6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:24.252147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae01abf11b6c49ac8cbd7e50e432e8a6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:24.254029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae01abf11b6c49ac8cbd7e50e432e8a6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:24.254988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae01abf11b6c49ac8cbd7e50e432e8a6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:24.286709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 847/1695] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:24 2022] DELETE /identity/v3/users/ae01abf11b6c49ac8cbd7e50e432e8a6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:24.287555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:24.290837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:24.297748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 849/1696] 10.222.0.71 () {58 vars in 1058 bytes} [Mon Jul 18 14:19:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:24.299832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8769d4-c3bb-4aa9-902f-ea6c755dc33a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.300657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8769d4-c3bb-4aa9-902f-ea6c755dc33a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.301212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8769d4-c3bb-4aa9-902f-ea6c755dc33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.340371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6a8769d4-c3bb-4aa9-902f-ea6c755dc33a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:24.342247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6a8769d4-c3bb-4aa9-902f-ea6c755dc33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:24.353897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:24.479406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c667e-db46-4383-82cd-bcd51cef6753 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:24.495011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:24.495713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.496002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.496283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.504675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:24.516385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 850/1697] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:24.517504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8769d4-c3bb-4aa9-902f-ea6c755dc33a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:24.521362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6a8769d4-c3bb-4aa9-902f-ea6c755dc33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:24.564372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8769d4-c3bb-4aa9-902f-ea6c755dc33a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:24.583254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b90cd-0f2b-4829-ab26-08aa47889a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:24.584872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b90cd-0f2b-4829-ab26-08aa47889a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.585365 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b90cd-0f2b-4829-ab26-08aa47889a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.585941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b90cd-0f2b-4829-ab26-08aa47889a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.587267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b90cd-0f2b-4829-ab26-08aa47889a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:24.588955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b90cd-0f2b-4829-ab26-08aa47889a4d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:24.633050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 851/1698] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:24.633050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7199bf-e591-478f-b409-7d8bcb08fd43 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.633050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7199bf-e591-478f-b409-7d8bcb08fd43 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.633050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7199bf-e591-478f-b409-7d8bcb08fd43 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.647488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 852/1699] 10.222.0.71 () {54 vars in 912 bytes} [Mon Jul 18 14:19:24 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:24.649343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7199bf-e591-478f-b409-7d8bcb08fd43 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:24.662867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:24.664534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:24.716520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 848/1700] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:24.717511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:24.739832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1343ba-a181-4fda-813a-0f02dee0d0e8 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:24.780097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:24.781378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.786630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.788936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.791250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:24.792731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:24.797360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:24.849495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 849/1701] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:24.850993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:24.887679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7199bf-e591-478f-b409-7d8bcb08fd43 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:24.895511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fe12c-9492-483c-a990-ffc00d4f56e7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:24.904875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b604aa-e7c3-40c1-be74-fb5a590d6e4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:24.905557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b604aa-e7c3-40c1-be74-fb5a590d6e4f service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.905868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b604aa-e7c3-40c1-be74-fb5a590d6e4f service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.906147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b604aa-e7c3-40c1-be74-fb5a590d6e4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.909874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e53b32-ac32-4446-8251-df1ae4192d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:24.914153 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e53b32-ac32-4446-8251-df1ae4192d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:24.914606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e53b32-ac32-4446-8251-df1ae4192d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:24.915693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e53b32-ac32-4446-8251-df1ae4192d20 admin admin] PATH_INFO: `/v3/projects/45e8cae4b6304e5dbf72e13d6608403e/users/86fe6730087443899e63db6a81a1a449/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:24.925910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-31b604aa-e7c3-40c1-be74-fb5a590d6e4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:24.962338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b604aa-e7c3-40c1-be74-fb5a590d6e4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:24.964621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b604aa-e7c3-40c1-be74-fb5a590d6e4f service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:24.981871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e53b32-ac32-4446-8251-df1ae4192d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e8cae4b6304e5dbf72e13d6608403e, user_id=86fe6730087443899e63db6a81a1a449, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:24.993767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 853/1702] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:24.997089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e53b32-ac32-4446-8251-df1ae4192d20 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.005426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e177-9538-4e56-8e42-be11ce0b1118 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.005994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e177-9538-4e56-8e42-be11ce0b1118 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.006291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8e177-9538-4e56-8e42-be11ce0b1118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.043596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 850/1703] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:24 2022] PUT /identity/v3/projects/45e8cae4b6304e5dbf72e13d6608403e/users/86fe6730087443899e63db6a81a1a449/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:25.044728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e53b32-ac32-4446-8251-df1ae4192d20 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:25.102607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:25.102828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bba8e177-9538-4e56-8e42-be11ce0b1118 None None] MFA Rules not processed for user `1f1fe084c3164e50bf5d904350925f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:25.104093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bba8e177-9538-4e56-8e42-be11ce0b1118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:25.225455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e53b32-ac32-4446-8251-df1ae4192d20 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:25.232856 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c56d4-2058-4c96-a8e3-680e9c0d5bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:25.233877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c56d4-2058-4c96-a8e3-680e9c0d5bf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.234525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c56d4-2058-4c96-a8e3-680e9c0d5bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.234945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c56d4-2058-4c96-a8e3-680e9c0d5bf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.235791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c56d4-2058-4c96-a8e3-680e9c0d5bf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:25.238611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c56d4-2058-4c96-a8e3-680e9c0d5bf1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.281309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 854/1704] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:19:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:25.281706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 851/1705] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:25 2022] POST /identity/v3/projects => generated 369 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:25.283325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a4572-a91b-4ab9-8a5b-fea93bfa245e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.283669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a4572-a91b-4ab9-8a5b-fea93bfa245e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.284829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a4572-a91b-4ab9-8a5b-fea93bfa245e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.288960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8e177-9538-4e56-8e42-be11ce0b1118 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:25.304291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8e177-9538-4e56-8e42-be11ce0b1118 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:25.342098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-693a4572-a91b-4ab9-8a5b-fea93bfa245e None None] MFA Rules not processed for user `86fe6730087443899e63db6a81a1a449`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:25.343554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-693a4572-a91b-4ab9-8a5b-fea93bfa245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:25.376349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:25.416695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:25.417449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.417765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.418182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.418892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:25.420940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.469909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 855/1706] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:25 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:25.470761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:25.480875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:25.527568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 852/1707] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:25.528900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a4572-a91b-4ab9-8a5b-fea93bfa245e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:25.573067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a4572-a91b-4ab9-8a5b-fea93bfa245e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:25.585252 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:25.586281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.586684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.586838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.587601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:25.590566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.605548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 853/1708] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:25.606739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:25.609404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:25.708941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe6947-9e1f-403a-bc49-786c87455c20 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:25.718472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc5004-125a-45c1-9a6d-880fed38d8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:25.720886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc5004-125a-45c1-9a6d-880fed38d8df service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.722028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc5004-125a-45c1-9a6d-880fed38d8df service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.722986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc5004-125a-45c1-9a6d-880fed38d8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.746161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc5004-125a-45c1-9a6d-880fed38d8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:25.750757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc5004-125a-45c1-9a6d-880fed38d8df service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.755436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a138bcc-d4ff-46f1-ac21-4cec39cdc001 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:25.779296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff1da4-30d9-4e63-bf6a-eec626814a12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:25.780170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff1da4-30d9-4e63-bf6a-eec626814a12 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.780555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff1da4-30d9-4e63-bf6a-eec626814a12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.781442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff1da4-30d9-4e63-bf6a-eec626814a12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.795343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 856/1709] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:25.797532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc5004-125a-45c1-9a6d-880fed38d8df service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:25.800269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff1da4-30d9-4e63-bf6a-eec626814a12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:25.805982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff1da4-30d9-4e63-bf6a-eec626814a12 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.831892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc5004-125a-45c1-9a6d-880fed38d8df service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:25.840701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 854/1710] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:25 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:25.855984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c7421-722c-4577-b69c-b76d2343f50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:25.857345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6b8ff-c315-4d2e-901e-14c9a2706548 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.857851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6b8ff-c315-4d2e-901e-14c9a2706548 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.860532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c7421-722c-4577-b69c-b76d2343f50f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.860910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c7421-722c-4577-b69c-b76d2343f50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.861203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c7421-722c-4577-b69c-b76d2343f50f admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.862077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c7421-722c-4577-b69c-b76d2343f50f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:25.866172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c7421-722c-4577-b69c-b76d2343f50f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.867005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe6b8ff-c315-4d2e-901e-14c9a2706548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.880130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 857/1711] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:25.885960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c7421-722c-4577-b69c-b76d2343f50f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:25.898233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c7421-722c-4577-b69c-b76d2343f50f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:25.903457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20977e-b5c3-4682-af07-b2576c6c4a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:25.904562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20977e-b5c3-4682-af07-b2576c6c4a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.904947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20977e-b5c3-4682-af07-b2576c6c4a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.905330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20977e-b5c3-4682-af07-b2576c6c4a7f admin admin] PATH_INFO: `/v3/projects/1f807109c12a48e492c65299a9e8bf21/users/bea9d5e106d742b19bbd7b3d80557e58/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.918494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0fe6b8ff-c315-4d2e-901e-14c9a2706548 None None] MFA Rules not processed for user `3aab24bab8d74b7aa5a20e7b2b89b5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:25.919769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0fe6b8ff-c315-4d2e-901e-14c9a2706548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:25.924768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20977e-b5c3-4682-af07-b2576c6c4a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f807109c12a48e492c65299a9e8bf21, user_id=bea9d5e106d742b19bbd7b3d80557e58, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:25.928813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20977e-b5c3-4682-af07-b2576c6c4a7f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:25.952195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 858/1712] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:25 2022] PUT /identity/v3/projects/1f807109c12a48e492c65299a9e8bf21/users/bea9d5e106d742b19bbd7b3d80557e58/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:19:25.964767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa5ce-94b8-4162-9cf9-3668c8cb9670 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:25.965217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa5ce-94b8-4162-9cf9-3668c8cb9670 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:25.965628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa5ce-94b8-4162-9cf9-3668c8cb9670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:25.991188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:26.023501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ad1fa5ce-94b8-4162-9cf9-3668c8cb9670 None None] MFA Rules not processed for user `bea9d5e106d742b19bbd7b3d80557e58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:26.027039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ad1fa5ce-94b8-4162-9cf9-3668c8cb9670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:26.076903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 855/1713] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:26.212861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 859/1714] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:26.218480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6b8ff-c315-4d2e-901e-14c9a2706548 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:26.289890 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe6b8ff-c315-4d2e-901e-14c9a2706548 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:26.656309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fb5a7-4390-45d8-a88f-273211fd4865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:26.657840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fb5a7-4390-45d8-a88f-273211fd4865 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:26.658153 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fb5a7-4390-45d8-a88f-273211fd4865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:26.658438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fb5a7-4390-45d8-a88f-273211fd4865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:26.662006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fb5a7-4390-45d8-a88f-273211fd4865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:26.664527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fb5a7-4390-45d8-a88f-273211fd4865 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:26.742849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 856/1715] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:26 2022] POST /identity/v3/projects => generated 373 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:26.746525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa5ce-94b8-4162-9cf9-3668c8cb9670 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:26.784932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa5ce-94b8-4162-9cf9-3668c8cb9670 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:26.882879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:26.935778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68fc8-1923-4c39-92f0-9d64d3a43c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:26.936523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68fc8-1923-4c39-92f0-9d64d3a43c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:26.936848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68fc8-1923-4c39-92f0-9d64d3a43c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:26.937133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68fc8-1923-4c39-92f0-9d64d3a43c70 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:26.938008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada68fc8-1923-4c39-92f0-9d64d3a43c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:26.940882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada68fc8-1923-4c39-92f0-9d64d3a43c70 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:26.982416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 860/1716] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:26 2022] POST /identity/v3/users => generated 293 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:26.987118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fb5a7-4390-45d8-a88f-273211fd4865 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:27.016164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fb5a7-4390-45d8-a88f-273211fd4865 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:27.026131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:27.027103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.027696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.028148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.030484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:27.034563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:27.061405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 857/1717] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:27.062387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68fc8-1923-4c39-92f0-9d64d3a43c70 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:27.080211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68fc8-1923-4c39-92f0-9d64d3a43c70 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:27.094716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:27.094921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c195a-0220-4663-a5fc-b0313fe93f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:27.096158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c195a-0220-4663-a5fc-b0313fe93f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.096449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c195a-0220-4663-a5fc-b0313fe93f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.096730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c195a-0220-4663-a5fc-b0313fe93f4b admin admin] PATH_INFO: `/v3/projects/4d73bf99f59742c383528c06073daa10/users/0eeeeea271134cd9b8ccdc75fcdf71f8/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.096975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:27.124584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76566c0-3ef6-4571-9db0-4ff2a28fbe70 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:27.134075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:27.136739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.137046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.137383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] PATH_INFO: `/v3/projects/0b76e7d662a340a29dd6667fcf933acd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.138806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b76e7d662a340a29dd6667fcf933acd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:27.139453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c195a-0220-4663-a5fc-b0313fe93f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d73bf99f59742c383528c06073daa10, user_id=0eeeeea271134cd9b8ccdc75fcdf71f8, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:27.141251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:27.142235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c195a-0220-4663-a5fc-b0313fe93f4b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:27.211183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 861/1718] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:27 2022] PUT /identity/v3/projects/4d73bf99f59742c383528c06073daa10/users/0eeeeea271134cd9b8ccdc75fcdf71f8/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:27.216605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c195a-0220-4663-a5fc-b0313fe93f4b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:27.300321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b76e7d662a340a29dd6667fcf933acd', 'request_id': 'req-cb38d936-bc34-48e2-92ad-a885db29307b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:27.330093 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b76e7d662a340a29dd6667fcf933acd', 'request_id': 'req-cb38d936-bc34-48e2-92ad-a885db29307b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:27.339519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b76e7d662a340a29dd6667fcf933acd', 'request_id': 'req-cb38d936-bc34-48e2-92ad-a885db29307b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:27.361356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] The token cache is being invalidate because project 0b76e7d662a340a29dd6667fcf933acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:27.361828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:27.362900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cb38d936-bc34-48e2-92ad-a885db29307b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:27.374053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 858/1719] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:27 2022] DELETE /identity/v3/projects/0b76e7d662a340a29dd6667fcf933acd => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:19:27.378497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690d71e-39c4-40ee-8d32-64cdaf5da4c9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.378497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690d71e-39c4-40ee-8d32-64cdaf5da4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.379226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690d71e-39c4-40ee-8d32-64cdaf5da4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.431893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1690d71e-39c4-40ee-8d32-64cdaf5da4c9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:27.434177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1690d71e-39c4-40ee-8d32-64cdaf5da4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:27.451691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c195a-0220-4663-a5fc-b0313fe93f4b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:27.506701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:27.598053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:27.598851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.599113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.599398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.600201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:27.602052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 859/1720] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:27.602697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:27.611612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea97ad2-8c28-4847-aad5-f2b4c7c6b957 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.612246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea97ad2-8c28-4847-aad5-f2b4c7c6b957 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.612824 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea97ad2-8c28-4847-aad5-f2b4c7c6b957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.618247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 862/1721] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:27.622154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:27.624209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:27.648601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116470e-ed4f-47db-91ad-cd17d9ff9a57 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:27.672621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cea97ad2-8c28-4847-aad5-f2b4c7c6b957 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:27.673917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cea97ad2-8c28-4847-aad5-f2b4c7c6b957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:27.680157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b144e5b-24cb-4ad0-8b6e-a72d22102a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:27.681200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b144e5b-24cb-4ad0-8b6e-a72d22102a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.684342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b144e5b-24cb-4ad0-8b6e-a72d22102a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.684784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b144e5b-24cb-4ad0-8b6e-a72d22102a6d admin admin] PATH_INFO: `/v3/projects/4d73bf99f59742c383528c06073daa10/users/0eeeeea271134cd9b8ccdc75fcdf71f8/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.696390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b144e5b-24cb-4ad0-8b6e-a72d22102a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d73bf99f59742c383528c06073daa10, user_id=0eeeeea271134cd9b8ccdc75fcdf71f8, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:27.696975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 860/1722] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:27.701099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea97ad2-8c28-4847-aad5-f2b4c7c6b957 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:27.704429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b144e5b-24cb-4ad0-8b6e-a72d22102a6d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:27.723532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea97ad2-8c28-4847-aad5-f2b4c7c6b957 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:27.736976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:27.747200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.747439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.748828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.751330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:27.751743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 863/1723] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:27 2022] PUT /identity/v3/projects/4d73bf99f59742c383528c06073daa10/users/0eeeeea271134cd9b8ccdc75fcdf71f8/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:27.762553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:27.764128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2a66a-b81e-4bed-bf68-9c53844671f7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:27.764559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2a66a-b81e-4bed-bf68-9c53844671f7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:27.764655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:27.765087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2a66a-b81e-4bed-bf68-9c53844671f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:27.791122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 861/1724] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:27.793837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:27.813570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-77a2a66a-b81e-4bed-bf68-9c53844671f7 None None] MFA Rules not processed for user `0eeeeea271134cd9b8ccdc75fcdf71f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:27.815714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-77a2a66a-b81e-4bed-bf68-9c53844671f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:27.819322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bbf82-5656-46c2-a2bd-12efe9b302da admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:27.845065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:28.079000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 864/1725] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:28.130976 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2a66a-b81e-4bed-bf68-9c53844671f7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:28.133901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a89ec-6dc8-487d-b27b-2223b7dda474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:28.136623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-77a2a66a-b81e-4bed-bf68-9c53844671f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:28.137574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a89ec-6dc8-487d-b27b-2223b7dda474 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:28.142238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a89ec-6dc8-487d-b27b-2223b7dda474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:28.143164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a89ec-6dc8-487d-b27b-2223b7dda474 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:28.144840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a89ec-6dc8-487d-b27b-2223b7dda474 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:28.146919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a89ec-6dc8-487d-b27b-2223b7dda474 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:28.258735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 862/1726] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:27 2022] POST /identity/v3/projects => generated 387 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:28.261164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a89ec-6dc8-487d-b27b-2223b7dda474 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:28.392773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2a66a-b81e-4bed-bf68-9c53844671f7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:28.462369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a89ec-6dc8-487d-b27b-2223b7dda474 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:28.503203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce506649-a016-4de0-9749-28c13533ddfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:28.504849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506649-a016-4de0-9749-28c13533ddfd admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:28.505163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506649-a016-4de0-9749-28c13533ddfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:28.508334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506649-a016-4de0-9749-28c13533ddfd admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:28.514292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce506649-a016-4de0-9749-28c13533ddfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:28.517092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce506649-a016-4de0-9749-28c13533ddfd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:28.596425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 863/1727] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:28 2022] POST /identity/v3/users => generated 294 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:28.602194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce506649-a016-4de0-9749-28c13533ddfd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:28.636586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee4d6f-e58d-46aa-bdfa-64661bfc5164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:28.637077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee4d6f-e58d-46aa-bdfa-64661bfc5164 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:28.637549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee4d6f-e58d-46aa-bdfa-64661bfc5164 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:28.638132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee4d6f-e58d-46aa-bdfa-64661bfc5164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:28.638553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce506649-a016-4de0-9749-28c13533ddfd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:28.666094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dead9-cae7-4b6f-8dc5-a4b0152f54c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:28.666986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dead9-cae7-4b6f-8dc5-a4b0152f54c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:28.667264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dead9-cae7-4b6f-8dc5-a4b0152f54c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:28.667550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dead9-cae7-4b6f-8dc5-a4b0152f54c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:28.668333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dead9-cae7-4b6f-8dc5-a4b0152f54c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:28.673453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754dead9-cae7-4b6f-8dc5-a4b0152f54c5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:28.725349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 864/1728] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:28.731051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dead9-cae7-4b6f-8dc5-a4b0152f54c5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:28.770724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754dead9-cae7-4b6f-8dc5-a4b0152f54c5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:28.793459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20434050-0184-4c68-a9b2-5ed8f6288b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:28.796435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20434050-0184-4c68-a9b2-5ed8f6288b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:28.797599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20434050-0184-4c68-a9b2-5ed8f6288b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:28.798136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20434050-0184-4c68-a9b2-5ed8f6288b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:28.801229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20434050-0184-4c68-a9b2-5ed8f6288b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:28.805462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20434050-0184-4c68-a9b2-5ed8f6288b96 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:28.824282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 865/1729] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:28.830293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20434050-0184-4c68-a9b2-5ed8f6288b96 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:28.861468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:28.883727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20434050-0184-4c68-a9b2-5ed8f6288b96 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:28.907394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ae261-d272-4bcf-9eea-3786bc551504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:28.913902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ae261-d272-4bcf-9eea-3786bc551504 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:28.915657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ae261-d272-4bcf-9eea-3786bc551504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:28.916397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ae261-d272-4bcf-9eea-3786bc551504 admin admin] PATH_INFO: `/v3/projects/085661d69266477fa5f289689725d174/users/f540834254444f4f95b54b03210af4d0/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:29.010606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee4d6f-e58d-46aa-bdfa-64661bfc5164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:29.015107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ae261-d272-4bcf-9eea-3786bc551504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=085661d69266477fa5f289689725d174, user_id=f540834254444f4f95b54b03210af4d0, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:29.017011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee4d6f-e58d-46aa-bdfa-64661bfc5164 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:29.041971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ae261-d272-4bcf-9eea-3786bc551504 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:29.093153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 865/1730] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 965 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:29.110620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 866/1731] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:28 2022] PUT /identity/v3/projects/085661d69266477fa5f289689725d174/users/f540834254444f4f95b54b03210af4d0/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:29.121770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d9c0-4a90-4aee-a3dc-3788f3b93c4e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:29.122122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d9c0-4a90-4aee-a3dc-3788f3b93c4e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:29.122355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d9c0-4a90-4aee-a3dc-3788f3b93c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:29.241884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e980d9c0-4a90-4aee-a3dc-3788f3b93c4e None None] MFA Rules not processed for user `f540834254444f4f95b54b03210af4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:29.243458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e980d9c0-4a90-4aee-a3dc-3788f3b93c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:29.251253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:29.259475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ae261-d272-4bcf-9eea-3786bc551504 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:29.470977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ae261-d272-4bcf-9eea-3786bc551504 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:29.485518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd011261-7168-4a1f-acc3-e700b62019b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:29.488469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011261-7168-4a1f-acc3-e700b62019b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:29.488760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011261-7168-4a1f-acc3-e700b62019b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:29.489058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011261-7168-4a1f-acc3-e700b62019b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:29.499008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cd011261-7168-4a1f-acc3-e700b62019b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:29.576639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 866/1732] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:29.578080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980d9c0-4a90-4aee-a3dc-3788f3b93c4e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:29.678023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd011261-7168-4a1f-acc3-e700b62019b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:29.681294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd011261-7168-4a1f-acc3-e700b62019b6 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:29.716193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 867/1733] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:29.718691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040100-956e-4189-95aa-e118e692b269 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:29.718985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040100-956e-4189-95aa-e118e692b269 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:29.719272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040100-956e-4189-95aa-e118e692b269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:29.869345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cc040100-956e-4189-95aa-e118e692b269 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:29.870673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cc040100-956e-4189-95aa-e118e692b269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:29.905999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980d9c0-4a90-4aee-a3dc-3788f3b93c4e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:29.929007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb849a02-077d-4e77-a1e7-fbc8fb088fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:29.930632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a02-077d-4e77-a1e7-fbc8fb088fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:29.935004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a02-077d-4e77-a1e7-fbc8fb088fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:29.935504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb849a02-077d-4e77-a1e7-fbc8fb088fb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:29.936333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb849a02-077d-4e77-a1e7-fbc8fb088fb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:29.947797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb849a02-077d-4e77-a1e7-fbc8fb088fb1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:30.052697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 867/1734] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:29 2022] POST /identity/v3/projects => generated 389 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:30.068024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef97c9-1c41-4229-aae6-05aa42dbf51b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:30.068751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef97c9-1c41-4229-aae6-05aa42dbf51b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:30.069280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef97c9-1c41-4229-aae6-05aa42dbf51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:30.216683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-27ef97c9-1c41-4229-aae6-05aa42dbf51b None None] MFA Rules not processed for user `a0210f69010b4529822bcdac1bdf8b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:30.221204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 868/1735] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:19:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:30.227967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc040100-956e-4189-95aa-e118e692b269 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:30.240176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-27ef97c9-1c41-4229-aae6-05aa42dbf51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:30.310135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc040100-956e-4189-95aa-e118e692b269 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:30.315328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:30.523737 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 868/1736] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:30.531412 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749951c6-33f1-45eb-939c-fdb51d41d201 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:30.535704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749951c6-33f1-45eb-939c-fdb51d41d201 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:30.537421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749951c6-33f1-45eb-939c-fdb51d41d201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:30.574257 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:30.586096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:30.586096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:30.586096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:30.586096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:30.586096 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:30.625912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 869/1737] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:30 2022] POST /identity/v3/users => generated 301 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:30.633333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:30.635910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:30.669432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-749951c6-33f1-45eb-939c-fdb51d41d201 None None] MFA Rules not processed for user `8a45cafadb824e08bd9e86fd0d5c4c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:30.670920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-749951c6-33f1-45eb-939c-fdb51d41d201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:30.843267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:30.959049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 869/1738] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:30.962315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749951c6-33f1-45eb-939c-fdb51d41d201 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:30.993892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749951c6-33f1-45eb-939c-fdb51d41d201 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:31.011341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf88a6-81c7-4746-a05c-a854f28eb84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:31.012206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf88a6-81c7-4746-a05c-a854f28eb84b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:31.013748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf88a6-81c7-4746-a05c-a854f28eb84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:31.015520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf88a6-81c7-4746-a05c-a854f28eb84b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:31.020363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf88a6-81c7-4746-a05c-a854f28eb84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:31.024986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf88a6-81c7-4746-a05c-a854f28eb84b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:31.027470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e800f54-bb4c-43b7-95c7-44db5afd3c5f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:31.076346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34571f5f-c1ee-4514-b95b-de63bf1720e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:31.077878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571f5f-c1ee-4514-b95b-de63bf1720e9 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:31.082382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 870/1739] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:31.087690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571f5f-c1ee-4514-b95b-de63bf1720e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:31.088890 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34571f5f-c1ee-4514-b95b-de63bf1720e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:31.089524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf88a6-81c7-4746-a05c-a854f28eb84b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:31.123625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf88a6-81c7-4746-a05c-a854f28eb84b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:31.138015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c218-3406-41bd-828a-f735edd4c671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:31.139140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c218-3406-41bd-828a-f735edd4c671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:31.141119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c218-3406-41bd-828a-f735edd4c671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:31.144102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992c218-3406-41bd-828a-f735edd4c671 admin admin] PATH_INFO: `/v3/projects/7114f96db2944cc6b878af860574a1ff/users/3c1f015d8df4477598fbd225b582e195/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:31.209397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c218-3406-41bd-828a-f735edd4c671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7114f96db2944cc6b878af860574a1ff, user_id=3c1f015d8df4477598fbd225b582e195, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:31.212268 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c992c218-3406-41bd-828a-f735edd4c671 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:31.236010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34571f5f-c1ee-4514-b95b-de63bf1720e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:31.239671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34571f5f-c1ee-4514-b95b-de63bf1720e9 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:31.264552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 870/1740] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:19:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:19:31.266464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 871/1741] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:31 2022] PUT /identity/v3/projects/7114f96db2944cc6b878af860574a1ff/users/3c1f015d8df4477598fbd225b582e195/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:31.271305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34571f5f-c1ee-4514-b95b-de63bf1720e9 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:31.331648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34571f5f-c1ee-4514-b95b-de63bf1720e9 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:31.403888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:31.616459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0981b3-08f9-42b8-97df-8c5085a5d2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:31.617353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0981b3-08f9-42b8-97df-8c5085a5d2fa admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:31.617786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0981b3-08f9-42b8-97df-8c5085a5d2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:31.618195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0981b3-08f9-42b8-97df-8c5085a5d2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:31.619117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0981b3-08f9-42b8-97df-8c5085a5d2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:31.621138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0981b3-08f9-42b8-97df-8c5085a5d2fa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:31.680640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 871/1742] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:31.682214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c218-3406-41bd-828a-f735edd4c671 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:31.714895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992c218-3406-41bd-828a-f735edd4c671 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:31.745824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:31.748665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:31.749276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:31.749893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] PATH_INFO: `/v3/projects/7114f96db2944cc6b878af860574a1ff/users/3c1f015d8df4477598fbd225b582e195/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:31.758968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=7114f96db2944cc6b878af860574a1ff, user_id=3c1f015d8df4477598fbd225b582e195, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:31.773220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:31.898812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 872/1743] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:31 2022] PUT /identity/v3/projects/7114f96db2944cc6b878af860574a1ff/users/3c1f015d8df4477598fbd225b582e195/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:31.909926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fdd05-8be8-44a6-ac85-607c28d9e3cf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:31.910225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fdd05-8be8-44a6-ac85-607c28d9e3cf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:31.910592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fdd05-8be8-44a6-ac85-607c28d9e3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:32.042760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7c0fdd05-8be8-44a6-ac85-607c28d9e3cf None None] MFA Rules not processed for user `3c1f015d8df4477598fbd225b582e195`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:32.044054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7c0fdd05-8be8-44a6-ac85-607c28d9e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:32.360135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 872/1744] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:32.376800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:32.381423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:32.834520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285205a7-b1aa-48b1-b905-e574e9ddf1ec admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:32.851950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b42c7-7c45-4c1e-95ac-7ee518743c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:32.852746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b42c7-7c45-4c1e-95ac-7ee518743c90 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:32.853064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b42c7-7c45-4c1e-95ac-7ee518743c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:32.853353 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b42c7-7c45-4c1e-95ac-7ee518743c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:32.919347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b42c7-7c45-4c1e-95ac-7ee518743c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:32.929058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b42c7-7c45-4c1e-95ac-7ee518743c90 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:32.992256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 873/1745] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:33.108696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15765f-8d1f-4147-addd-66861e29eb01 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:33.108992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15765f-8d1f-4147-addd-66861e29eb01 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:33.109276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15765f-8d1f-4147-addd-66861e29eb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:33.173522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9a15765f-8d1f-4147-addd-66861e29eb01 None None] MFA Rules not processed for user `5ca8be8093b547ee953ea646f90fdcae`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:33.183025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9a15765f-8d1f-4147-addd-66861e29eb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:33.361076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b42c7-7c45-4c1e-95ac-7ee518743c90 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:33.457974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 873/1746] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:33.545141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b42c7-7c45-4c1e-95ac-7ee518743c90 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:33.566036 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:33.567271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:33.567723 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:33.568445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] PATH_INFO: `/v3/users/f540834254444f4f95b54b03210af4d0` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:33.570028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f540834254444f4f95b54b03210af4d0)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:33.572256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:33.721973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:33.765286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f540834254444f4f95b54b03210af4d0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:33.783964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f540834254444f4f95b54b03210af4d0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:33.820070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f540834254444f4f95b54b03210af4d0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:33.821107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f540834254444f4f95b54b03210af4d0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:33.821886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dae5d9b4-2476-4a84-95e4-2d6002c7ce1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f540834254444f4f95b54b03210af4d0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:33.889783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 874/1747] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:33 2022] DELETE /identity/v3/users/f540834254444f4f95b54b03210af4d0 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:33.896190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ebfca-2a55-4557-857e-11664a9b9aed None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:33.896565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ebfca-2a55-4557-857e-11664a9b9aed None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:33.896761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64ebfca-2a55-4557-857e-11664a9b9aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:33.996880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d64ebfca-2a55-4557-857e-11664a9b9aed None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:34.001751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d64ebfca-2a55-4557-857e-11664a9b9aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:34.071814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866745e4-bd91-4ef9-a981-453fbc034230 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:34.072494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866745e4-bd91-4ef9-a981-453fbc034230 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:34.072999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866745e4-bd91-4ef9-a981-453fbc034230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:34.151965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-866745e4-bd91-4ef9-a981-453fbc034230 None None] MFA Rules not processed for user `1f1fe084c3164e50bf5d904350925f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:34.155822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-866745e4-bd91-4ef9-a981-453fbc034230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:34.193616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:34.303592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 874/1748] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:34.366596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ebfca-2a55-4557-857e-11664a9b9aed None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:34.367811 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d64ebfca-2a55-4557-857e-11664a9b9aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:34.448725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 875/1749] 10.222.0.104 () {58 vars in 1016 bytes} [Mon Jul 18 14:19:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:34.536612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64ebfca-2a55-4557-857e-11664a9b9aed None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:34.552845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:34.553555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:34.553872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:34.554159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:34.598400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:34.600632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:34.638949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 875/1750] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:19:34.838593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866745e4-bd91-4ef9-a981-453fbc034230 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:34.843197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-866745e4-bd91-4ef9-a981-453fbc034230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:34.895820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866745e4-bd91-4ef9-a981-453fbc034230 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:34.916300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad36ee0-029a-4af9-bb51-eeb74d469330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:34.917410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad36ee0-029a-4af9-bb51-eeb74d469330 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:34.919083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad36ee0-029a-4af9-bb51-eeb74d469330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:34.919658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad36ee0-029a-4af9-bb51-eeb74d469330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:34.944255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad36ee0-029a-4af9-bb51-eeb74d469330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:34.951345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad36ee0-029a-4af9-bb51-eeb74d469330 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:34.980016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 876/1751] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:37.834563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:37.836710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:37.888928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfa658-1930-4d78-a076-3d34c15c3b1a service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:37.917236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:37.918514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:37.918823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:37.919115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] PATH_INFO: `/v3/projects/085661d69266477fa5f289689725d174` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:37.922487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=085661d69266477fa5f289689725d174)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:37.935951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:38.119790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '085661d69266477fa5f289689725d174', 'request_id': 'req-24c43886-e1dd-4ca5-9747-48a41d9a0485'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.134403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '085661d69266477fa5f289689725d174', 'request_id': 'req-24c43886-e1dd-4ca5-9747-48a41d9a0485'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.149876 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '085661d69266477fa5f289689725d174', 'request_id': 'req-24c43886-e1dd-4ca5-9747-48a41d9a0485'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.208350 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] The token cache is being invalidate because project 085661d69266477fa5f289689725d174 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:38.208665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.209448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-24c43886-e1dd-4ca5-9747-48a41d9a0485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.211150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 876/1752] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:37 2022] DELETE /identity/v3/projects/085661d69266477fa5f289689725d174 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:38.215577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad36ee0-029a-4af9-bb51-eeb74d469330 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:38.241733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad36ee0-029a-4af9-bb51-eeb74d469330 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:38.467193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:38.468363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:38.468973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:38.470037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] PATH_INFO: `/v3/users/3c1f015d8df4477598fbd225b582e195` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:38.475039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1f015d8df4477598fbd225b582e195)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:38.477390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:38.665340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1f015d8df4477598fbd225b582e195'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.675482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:38.687585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1f015d8df4477598fbd225b582e195'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.745794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1f015d8df4477598fbd225b582e195'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.748687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1f015d8df4477598fbd225b582e195'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.750059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2b1da7ba-d788-4519-973c-a2a910599bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1f015d8df4477598fbd225b582e195'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:38.786705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 877/1753] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:38 2022] DELETE /identity/v3/users/3c1f015d8df4477598fbd225b582e195 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:38.964525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bdb1e-dc4a-4392-9da4-0d0d8711ea97 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:38.965423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bdb1e-dc4a-4392-9da4-0d0d8711ea97 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:38.965913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686bdb1e-dc4a-4392-9da4-0d0d8711ea97 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:38.969990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 877/1754] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:19:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:39.473560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fcb6c-d385-400e-a8d7-f7289885eb2a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:39.474234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fcb6c-d385-400e-a8d7-f7289885eb2a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:39.474554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37fcb6c-d385-400e-a8d7-f7289885eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:39.536750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f37fcb6c-d385-400e-a8d7-f7289885eb2a None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:39.538578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f37fcb6c-d385-400e-a8d7-f7289885eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:39.850407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 878/1755] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:19:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:41.520250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bdb1e-dc4a-4392-9da4-0d0d8711ea97 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:41.522814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-686bdb1e-dc4a-4392-9da4-0d0d8711ea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:41.598468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686bdb1e-dc4a-4392-9da4-0d0d8711ea97 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:41.624095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d20048-78ca-4fad-bf47-e512593fb2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:41.625572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d20048-78ca-4fad-bf47-e512593fb2bc service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:41.627327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d20048-78ca-4fad-bf47-e512593fb2bc service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:41.628495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d20048-78ca-4fad-bf47-e512593fb2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:41.657470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d20048-78ca-4fad-bf47-e512593fb2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:41.661003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d20048-78ca-4fad-bf47-e512593fb2bc service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:41.725213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 878/1756] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:43.330075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fcb6c-d385-400e-a8d7-f7289885eb2a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:43.332510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f37fcb6c-d385-400e-a8d7-f7289885eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:43.627445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37fcb6c-d385-400e-a8d7-f7289885eb2a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:43.653205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:43.660232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:43.661238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:43.663557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] PATH_INFO: `/v3/projects/7114f96db2944cc6b878af860574a1ff` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:43.668994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7114f96db2944cc6b878af860574a1ff)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:43.674608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:43.834398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7114f96db2944cc6b878af860574a1ff', 'request_id': 'req-182e33be-63b8-4697-bba7-926d6f0b9f1c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:43.860681 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7114f96db2944cc6b878af860574a1ff', 'request_id': 'req-182e33be-63b8-4697-bba7-926d6f0b9f1c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:43.888347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7114f96db2944cc6b878af860574a1ff', 'request_id': 'req-182e33be-63b8-4697-bba7-926d6f0b9f1c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:43.890913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:43.914612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] The token cache is being invalidate because project 7114f96db2944cc6b878af860574a1ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:43.914972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:43.915837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-182e33be-63b8-4697-bba7-926d6f0b9f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:43.918028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 879/1757] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:43 2022] DELETE /identity/v3/projects/7114f96db2944cc6b878af860574a1ff => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:43.923057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049960c-79bc-4e98-a5a6-d28db3269f50 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:43.925013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049960c-79bc-4e98-a5a6-d28db3269f50 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:43.925879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049960c-79bc-4e98-a5a6-d28db3269f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:44.013580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2049960c-79bc-4e98-a5a6-d28db3269f50 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:44.044315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2049960c-79bc-4e98-a5a6-d28db3269f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:44.282149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:44.317421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 879/1758] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:44.326500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2013c-4791-4156-b707-e9bcaf0b8d3a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:44.326788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2013c-4791-4156-b707-e9bcaf0b8d3a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:44.326914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2013c-4791-4156-b707-e9bcaf0b8d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:44.371791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-53f2013c-4791-4156-b707-e9bcaf0b8d3a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:44.373022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-53f2013c-4791-4156-b707-e9bcaf0b8d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:44.392282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 880/1759] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:44.395331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049960c-79bc-4e98-a5a6-d28db3269f50 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:44.409179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049960c-79bc-4e98-a5a6-d28db3269f50 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:44.425362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:44.426152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:44.426430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:44.426710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:44.427521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:44.428466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:44.432849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:44.468076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 880/1760] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:44.479337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2013c-4791-4156-b707-e9bcaf0b8d3a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:44.503938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2013c-4791-4156-b707-e9bcaf0b8d3a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:44.533311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7e474-a456-4a25-a3ea-046e23eb4f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:44.534095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7e474-a456-4a25-a3ea-046e23eb4f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:44.534381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7e474-a456-4a25-a3ea-046e23eb4f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:44.534661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7e474-a456-4a25-a3ea-046e23eb4f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:44.539276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7e474-a456-4a25-a3ea-046e23eb4f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:44.546154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7e474-a456-4a25-a3ea-046e23eb4f7e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:44.676240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 881/1761] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:44 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:44.678880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:44.704080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7b2fd-5f7e-4f43-be22-fe9a2721d522 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:44.948463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4f79-3623-49c0-942d-8241a65c3acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:44.950541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4f79-3623-49c0-942d-8241a65c3acf admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:44.952217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4f79-3623-49c0-942d-8241a65c3acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:44.952976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4f79-3623-49c0-942d-8241a65c3acf admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:44.955504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4f79-3623-49c0-942d-8241a65c3acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:44.971038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4f79-3623-49c0-942d-8241a65c3acf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:45.038229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 881/1762] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:44 2022] POST /identity/v3/users => generated 294 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:45.042425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7e474-a456-4a25-a3ea-046e23eb4f7e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:45.061805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7e474-a456-4a25-a3ea-046e23eb4f7e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:45.098748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f4a29-2959-45cb-82d7-83fbab235669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:45.102371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f4a29-2959-45cb-82d7-83fbab235669 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:45.102371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f4a29-2959-45cb-82d7-83fbab235669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:45.102371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f4a29-2959-45cb-82d7-83fbab235669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:45.102371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f4a29-2959-45cb-82d7-83fbab235669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:45.114993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f4a29-2959-45cb-82d7-83fbab235669 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:45.162076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 882/1763] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:45.167083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4f79-3623-49c0-942d-8241a65c3acf admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:45.196135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4f79-3623-49c0-942d-8241a65c3acf admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:45.204209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35ed79-9098-4bb7-8645-77d661c4475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:45.205029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35ed79-9098-4bb7-8645-77d661c4475a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:45.205376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35ed79-9098-4bb7-8645-77d661c4475a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:45.205835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35ed79-9098-4bb7-8645-77d661c4475a admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:45.207117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35ed79-9098-4bb7-8645-77d661c4475a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:45.208939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35ed79-9098-4bb7-8645-77d661c4475a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:45.237143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 882/1764] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:45 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:45.243019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f4a29-2959-45cb-82d7-83fbab235669 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:45.292043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f4a29-2959-45cb-82d7-83fbab235669 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:45.326183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52daacd-fe33-447d-98c5-86ffff066d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:45.327461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52daacd-fe33-447d-98c5-86ffff066d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:45.327919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52daacd-fe33-447d-98c5-86ffff066d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:45.328354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52daacd-fe33-447d-98c5-86ffff066d0d admin admin] PATH_INFO: `/v3/projects/20fd23b3580a42c2958e8d6f667f05cd/users/ac489f41a7b1493090529bac7945307e/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:45.358618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52daacd-fe33-447d-98c5-86ffff066d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fd23b3580a42c2958e8d6f667f05cd, user_id=ac489f41a7b1493090529bac7945307e, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:45.362419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52daacd-fe33-447d-98c5-86ffff066d0d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:45.428305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 883/1765] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:45 2022] PUT /identity/v3/projects/20fd23b3580a42c2958e8d6f667f05cd/users/ac489f41a7b1493090529bac7945307e/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:45.454027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6b790-55da-4ac9-93bf-9fd5372ad1ca None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:45.455098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6b790-55da-4ac9-93bf-9fd5372ad1ca None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:45.455978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6b790-55da-4ac9-93bf-9fd5372ad1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:45.529379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f9e6b790-55da-4ac9-93bf-9fd5372ad1ca None None] MFA Rules not processed for user `ac489f41a7b1493090529bac7945307e`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:45.531161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f9e6b790-55da-4ac9-93bf-9fd5372ad1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:45.812203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52daacd-fe33-447d-98c5-86ffff066d0d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:45.813303 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 883/1766] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:46.079876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52daacd-fe33-447d-98c5-86ffff066d0d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:46.092387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d507a9-1870-40b5-bda2-758387638d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:46.092655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d507a9-1870-40b5-bda2-758387638d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:46.092941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d507a9-1870-40b5-bda2-758387638d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:46.093396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d507a9-1870-40b5-bda2-758387638d01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:46.094479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d507a9-1870-40b5-bda2-758387638d01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:46.120723 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d507a9-1870-40b5-bda2-758387638d01 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:46.237045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 884/1767] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:45 2022] POST /identity/v3/projects => generated 385 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:46.239386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6b790-55da-4ac9-93bf-9fd5372ad1ca None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:46.266307 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6b790-55da-4ac9-93bf-9fd5372ad1ca None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:46.416807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:46.443211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c867f3-8dd6-440b-8a25-4cc2a37f54d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:46.444094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c867f3-8dd6-440b-8a25-4cc2a37f54d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:46.444876 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c867f3-8dd6-440b-8a25-4cc2a37f54d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:46.445288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c867f3-8dd6-440b-8a25-4cc2a37f54d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:46.446506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c867f3-8dd6-440b-8a25-4cc2a37f54d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:46.455333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c867f3-8dd6-440b-8a25-4cc2a37f54d9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:46.511279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 884/1768] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:46 2022] POST /identity/v3/users => generated 299 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:46.517336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d507a9-1870-40b5-bda2-758387638d01 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:46.553011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d507a9-1870-40b5-bda2-758387638d01 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:46.566014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1bdf2-91e2-479b-9b43-1ca2693ca323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:46.567936 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1bdf2-91e2-479b-9b43-1ca2693ca323 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:46.568763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1bdf2-91e2-479b-9b43-1ca2693ca323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:46.569478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1bdf2-91e2-479b-9b43-1ca2693ca323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:46.571449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1bdf2-91e2-479b-9b43-1ca2693ca323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:46.575968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1bdf2-91e2-479b-9b43-1ca2693ca323 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:46.598279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 885/1769] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:46.603100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c867f3-8dd6-440b-8a25-4cc2a37f54d9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:46.649044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c867f3-8dd6-440b-8a25-4cc2a37f54d9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:46.666292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa944f-0168-4a62-bc17-bc9fa701efe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:46.667721 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa944f-0168-4a62-bc17-bc9fa701efe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:46.668385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa944f-0168-4a62-bc17-bc9fa701efe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:46.668886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa944f-0168-4a62-bc17-bc9fa701efe7 admin admin] PATH_INFO: `/v3/projects/cad15b37fc244014a20502dcc03ab347/users/a2e7717158c140febffbdfb40e3b0a11/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:46.713466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa944f-0168-4a62-bc17-bc9fa701efe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad15b37fc244014a20502dcc03ab347, user_id=a2e7717158c140febffbdfb40e3b0a11, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:46.721861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa944f-0168-4a62-bc17-bc9fa701efe7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:46.755058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 885/1770] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:46 2022] PUT /identity/v3/projects/cad15b37fc244014a20502dcc03ab347/users/a2e7717158c140febffbdfb40e3b0a11/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:46.758901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1bdf2-91e2-479b-9b43-1ca2693ca323 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:46.772137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1bdf2-91e2-479b-9b43-1ca2693ca323 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:46.845622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5b379-9814-424e-b00b-04b05e6ecea9 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:46.846654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5b379-9814-424e-b00b-04b05e6ecea9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:46.847542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5b379-9814-424e-b00b-04b05e6ecea9 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:46.849772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 886/1771] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:19:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:46.875747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61584e3-a7e2-4f74-a527-25281865718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:46.876614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61584e3-a7e2-4f74-a527-25281865718f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:46.877342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61584e3-a7e2-4f74-a527-25281865718f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:46.877865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61584e3-a7e2-4f74-a527-25281865718f admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:46.878854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61584e3-a7e2-4f74-a527-25281865718f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:46.881635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61584e3-a7e2-4f74-a527-25281865718f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:46.884394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:46.887692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d94c6-111d-4995-9447-4acedb38cb78 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:46.888982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d94c6-111d-4995-9447-4acedb38cb78 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:46.889389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d94c6-111d-4995-9447-4acedb38cb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:46.919534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 886/1772] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:46.923982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61584e3-a7e2-4f74-a527-25281865718f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:46.925764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-912d94c6-111d-4995-9447-4acedb38cb78 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:46.933043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-912d94c6-111d-4995-9447-4acedb38cb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:46.945420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61584e3-a7e2-4f74-a527-25281865718f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:46.963639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b864fb6-ea37-40c4-9093-2d858f118e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:46.964722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b864fb6-ea37-40c4-9093-2d858f118e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:46.965104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b864fb6-ea37-40c4-9093-2d858f118e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:46.965513 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b864fb6-ea37-40c4-9093-2d858f118e27 admin admin] PATH_INFO: `/v3/projects/cad15b37fc244014a20502dcc03ab347/users/a2e7717158c140febffbdfb40e3b0a11/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:46.972267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b864fb6-ea37-40c4-9093-2d858f118e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad15b37fc244014a20502dcc03ab347, user_id=a2e7717158c140febffbdfb40e3b0a11, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:46.977747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b864fb6-ea37-40c4-9093-2d858f118e27 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:47.019731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 887/1773] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:46 2022] PUT /identity/v3/projects/cad15b37fc244014a20502dcc03ab347/users/a2e7717158c140febffbdfb40e3b0a11/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:47.031939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4717a-e87a-406d-ac27-33a4dde8819c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:47.032345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4717a-e87a-406d-ac27-33a4dde8819c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:47.032813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4717a-e87a-406d-ac27-33a4dde8819c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:47.109843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-26c4717a-e87a-406d-ac27-33a4dde8819c None None] MFA Rules not processed for user `a2e7717158c140febffbdfb40e3b0a11`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:47.113434 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-26c4717a-e87a-406d-ac27-33a4dde8819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:47.174201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 887/1774] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:19:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:47.252574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 888/1775] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:47.267470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d94c6-111d-4995-9447-4acedb38cb78 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:47.269297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-912d94c6-111d-4995-9447-4acedb38cb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:47.347609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d94c6-111d-4995-9447-4acedb38cb78 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:47.389522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4717a-e87a-406d-ac27-33a4dde8819c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:47.391086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-26c4717a-e87a-406d-ac27-33a4dde8819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:47.456648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fde6e-15b1-49c5-b96d-f2ab7dcf8d0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:47.457435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fde6e-15b1-49c5-b96d-f2ab7dcf8d0d service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:47.457759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fde6e-15b1-49c5-b96d-f2ab7dcf8d0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:47.458158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fde6e-15b1-49c5-b96d-f2ab7dcf8d0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:47.471853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fde6e-15b1-49c5-b96d-f2ab7dcf8d0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:47.473903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fde6e-15b1-49c5-b96d-f2ab7dcf8d0d service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:47.487649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 888/1776] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:47.560685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4717a-e87a-406d-ac27-33a4dde8819c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:47.592626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df807ca5-c8e1-4d45-b869-738bbe171698 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:47.594058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807ca5-c8e1-4d45-b869-738bbe171698 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:47.594266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807ca5-c8e1-4d45-b869-738bbe171698 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:47.595683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df807ca5-c8e1-4d45-b869-738bbe171698 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:47.616130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df807ca5-c8e1-4d45-b869-738bbe171698 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:47.619174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df807ca5-c8e1-4d45-b869-738bbe171698 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:47.648212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 889/1777] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:48.478323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fde6e-15b1-49c5-b96d-f2ab7dcf8d0d service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:48.493774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:48.622378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fde6e-15b1-49c5-b96d-f2ab7dcf8d0d service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:48.629481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:48.630587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:48.631030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:48.631615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] PATH_INFO: `/v3/users/ac489f41a7b1493090529bac7945307e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:48.633386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac489f41a7b1493090529bac7945307e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:48.637357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:48.705879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac489f41a7b1493090529bac7945307e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:48.716357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac489f41a7b1493090529bac7945307e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:48.735636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac489f41a7b1493090529bac7945307e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:48.736423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac489f41a7b1493090529bac7945307e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:48.737102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac489f41a7b1493090529bac7945307e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:48.750663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 889/1778] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:48 2022] DELETE /identity/v3/users/ac489f41a7b1493090529bac7945307e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:48.755774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb858ce3-532f-41de-b72b-5f94845a5352 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:48.756318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb858ce3-532f-41de-b72b-5f94845a5352 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:48.756447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb858ce3-532f-41de-b72b-5f94845a5352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:48.775817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bb858ce3-532f-41de-b72b-5f94845a5352 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:48.777184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bb858ce3-532f-41de-b72b-5f94845a5352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:48.852824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:48.878987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 890/1779] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:48.887165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:48.893148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:48.982898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1e620-fbf2-437a-906c-0f768f9b2df4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:48.987540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ede275-d04e-46ba-ad2b-a2b41d4d53b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:48.991946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ede275-d04e-46ba-ad2b-a2b41d4d53b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:48.991946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ede275-d04e-46ba-ad2b-a2b41d4d53b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:48.991946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ede275-d04e-46ba-ad2b-a2b41d4d53b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:49.008011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ede275-d04e-46ba-ad2b-a2b41d4d53b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:49.009588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ede275-d04e-46ba-ad2b-a2b41d4d53b5 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:49.026229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 890/1780] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:49.488964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb858ce3-532f-41de-b72b-5f94845a5352 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:49.490685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bb858ce3-532f-41de-b72b-5f94845a5352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:49.504515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb858ce3-532f-41de-b72b-5f94845a5352 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:49.514767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:49.516495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:49.517310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:49.517839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] PATH_INFO: `/v3/projects/20fd23b3580a42c2958e8d6f667f05cd` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:49.519542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20fd23b3580a42c2958e8d6f667f05cd)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:49.522105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:49.663871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20fd23b3580a42c2958e8d6f667f05cd', 'request_id': 'req-14ce0f9b-522c-4395-a255-75cd2fd8a803'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.693883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20fd23b3580a42c2958e8d6f667f05cd', 'request_id': 'req-14ce0f9b-522c-4395-a255-75cd2fd8a803'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.712151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20fd23b3580a42c2958e8d6f667f05cd', 'request_id': 'req-14ce0f9b-522c-4395-a255-75cd2fd8a803'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.729416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] The token cache is being invalidate because project 20fd23b3580a42c2958e8d6f667f05cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:49.729995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.730888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.735760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 891/1781] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:49 2022] DELETE /identity/v3/projects/20fd23b3580a42c2958e8d6f667f05cd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:49.737602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ede275-d04e-46ba-ad2b-a2b41d4d53b5 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:49.749371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ede275-d04e-46ba-ad2b-a2b41d4d53b5 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:49.785466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:49.821507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:49.823301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:49.823457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:49.823809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] PATH_INFO: `/v3/users/a2e7717158c140febffbdfb40e3b0a11` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:49.825243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e7717158c140febffbdfb40e3b0a11)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:49.827487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:49.920497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e7717158c140febffbdfb40e3b0a11'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.931696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e7717158c140febffbdfb40e3b0a11'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.960616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e7717158c140febffbdfb40e3b0a11'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.966740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e7717158c140febffbdfb40e3b0a11'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.967536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-82cd158a-b4be-46b0-be50-458b74745f3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e7717158c140febffbdfb40e3b0a11'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:49.998004 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 891/1782] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:49 2022] DELETE /identity/v3/users/a2e7717158c140febffbdfb40e3b0a11 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:50.761981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:50.763816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:50.806749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce0f9b-522c-4395-a255-75cd2fd8a803 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:50.887701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:50.888594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:50.888879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:50.889156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] PATH_INFO: `/v3/projects/cad15b37fc244014a20502dcc03ab347` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:50.890411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad15b37fc244014a20502dcc03ab347)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:50.892425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:50.947980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad15b37fc244014a20502dcc03ab347', 'request_id': 'req-73d07ce6-d03f-4af8-9e20-9708897a5e54'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:50.954985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad15b37fc244014a20502dcc03ab347', 'request_id': 'req-73d07ce6-d03f-4af8-9e20-9708897a5e54'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:50.964815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad15b37fc244014a20502dcc03ab347', 'request_id': 'req-73d07ce6-d03f-4af8-9e20-9708897a5e54'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:50.977735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] The token cache is being invalidate because project cad15b37fc244014a20502dcc03ab347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:50.977859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:50.978613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-73d07ce6-d03f-4af8-9e20-9708897a5e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:50.980364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 892/1783] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:50 2022] DELETE /identity/v3/projects/cad15b37fc244014a20502dcc03ab347 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:50.985800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca01a59-e888-4708-a9d9-d17819b74d5b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:50.986324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca01a59-e888-4708-a9d9-d17819b74d5b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:50.986324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca01a59-e888-4708-a9d9-d17819b74d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:51.013560 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-aca01a59-e888-4708-a9d9-d17819b74d5b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:51.016666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-aca01a59-e888-4708-a9d9-d17819b74d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:51.125362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 892/1784] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:51.128319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f5a38-69dd-41b3-aec7-60f57ef05158 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:51.128955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f5a38-69dd-41b3-aec7-60f57ef05158 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:51.129940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f5a38-69dd-41b3-aec7-60f57ef05158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:51.196413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-581f5a38-69dd-41b3-aec7-60f57ef05158 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:51.198300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-581f5a38-69dd-41b3-aec7-60f57ef05158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:51.212878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 893/1785] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:51.214320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca01a59-e888-4708-a9d9-d17819b74d5b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:51.240518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca01a59-e888-4708-a9d9-d17819b74d5b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:51.261919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:51.262666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:51.262951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:51.263239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:51.263968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:51.264278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:51.266087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:51.293628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 893/1786] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:19:51.295555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f5a38-69dd-41b3-aec7-60f57ef05158 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:51.302536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:51.327255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f5a38-69dd-41b3-aec7-60f57ef05158 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:51.344783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eb2c6-ea27-48ee-b17e-035a3786c689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:51.348071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eb2c6-ea27-48ee-b17e-035a3786c689 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:51.358809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eb2c6-ea27-48ee-b17e-035a3786c689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:51.363010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76eb2c6-ea27-48ee-b17e-035a3786c689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:51.368737 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76eb2c6-ea27-48ee-b17e-035a3786c689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:51.385996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76eb2c6-ea27-48ee-b17e-035a3786c689 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:51.520776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:51.521266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 894/1787] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:51 2022] POST /identity/v3/projects => generated 371 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:51.539382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc829d40-33d5-4b8b-8938-2b24a9cf58be admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:51.674861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:51.769007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457000c8-391e-40b1-bead-0c169b299b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:51.769865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457000c8-391e-40b1-bead-0c169b299b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:51.770831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457000c8-391e-40b1-bead-0c169b299b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:51.771213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457000c8-391e-40b1-bead-0c169b299b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:51.772331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457000c8-391e-40b1-bead-0c169b299b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:51.776620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457000c8-391e-40b1-bead-0c169b299b1a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:51.836450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 894/1788] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:51 2022] POST /identity/v3/users => generated 286 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:51.840772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eb2c6-ea27-48ee-b17e-035a3786c689 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:51.867404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76eb2c6-ea27-48ee-b17e-035a3786c689 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:51.882887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c76fc7-5935-42b2-9c9b-dd11718564bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:51.883586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76fc7-5935-42b2-9c9b-dd11718564bb admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:51.884033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76fc7-5935-42b2-9c9b-dd11718564bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:51.884499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c76fc7-5935-42b2-9c9b-dd11718564bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:51.885261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c76fc7-5935-42b2-9c9b-dd11718564bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:51.887323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c76fc7-5935-42b2-9c9b-dd11718564bb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:51.923746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 895/1789] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:51.928343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457000c8-391e-40b1-bead-0c169b299b1a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:51.945912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457000c8-391e-40b1-bead-0c169b299b1a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:51.957871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aacca0-e053-4342-b56c-d8b84c821d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:51.974924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aacca0-e053-4342-b56c-d8b84c821d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:51.976161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aacca0-e053-4342-b56c-d8b84c821d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:51.977589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aacca0-e053-4342-b56c-d8b84c821d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:51.986660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aacca0-e053-4342-b56c-d8b84c821d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:52.032954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aacca0-e053-4342-b56c-d8b84c821d1d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:52.075190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 895/1790] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:52.079356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c76fc7-5935-42b2-9c9b-dd11718564bb admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:52.094205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c76fc7-5935-42b2-9c9b-dd11718564bb admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:52.104123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:52.105335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:52.105803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:52.106261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] PATH_INFO: `/v3/projects/f12604fad21641819b829c553b63398b/users/c26e061da2844459990e1f0a79869468/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:52.146945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f12604fad21641819b829c553b63398b, user_id=c26e061da2844459990e1f0a79869468, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:52.151117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:52.186154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 896/1791] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:52 2022] PUT /identity/v3/projects/f12604fad21641819b829c553b63398b/users/c26e061da2844459990e1f0a79869468/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:52.192131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d47a4-b931-4e5a-bd8d-f612224cbc49 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:52.192439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d47a4-b931-4e5a-bd8d-f612224cbc49 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:52.195299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d47a4-b931-4e5a-bd8d-f612224cbc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:52.228133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f69d47a4-b931-4e5a-bd8d-f612224cbc49 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:52.229390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f69d47a4-b931-4e5a-bd8d-f612224cbc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:52.364539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 896/1792] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:52.405841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:52.414189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:52.532101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488d91a-1d8c-4dbd-9e73-0747db0d4341 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:52.563919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619abb79-fcb9-4556-b48a-d021f6c66f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:52.566753 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619abb79-fcb9-4556-b48a-d021f6c66f3f service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:52.566753 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619abb79-fcb9-4556-b48a-d021f6c66f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:52.566753 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619abb79-fcb9-4556-b48a-d021f6c66f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:52.613922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619abb79-fcb9-4556-b48a-d021f6c66f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:52.621214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619abb79-fcb9-4556-b48a-d021f6c66f3f service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:52.656848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 897/1793] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:52.760958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d47a4-b931-4e5a-bd8d-f612224cbc49 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:52.763285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f69d47a4-b931-4e5a-bd8d-f612224cbc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:52.785331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d47a4-b931-4e5a-bd8d-f612224cbc49 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:52.805116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:52.806056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:52.806343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:52.806621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] PATH_INFO: `/v3/users/86fe6730087443899e63db6a81a1a449` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:52.807917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86fe6730087443899e63db6a81a1a449)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:52.809846 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:52.902514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86fe6730087443899e63db6a81a1a449'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:52.921623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86fe6730087443899e63db6a81a1a449'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:52.951704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86fe6730087443899e63db6a81a1a449'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:52.952766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86fe6730087443899e63db6a81a1a449'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:52.953473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86fe6730087443899e63db6a81a1a449'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:53.003371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 897/1794] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:52 2022] DELETE /identity/v3/users/86fe6730087443899e63db6a81a1a449 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:53.008217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24cf67-329d-49c6-ba9f-fdff1e59c267 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:53.008712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24cf67-329d-49c6-ba9f-fdff1e59c267 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:53.009989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a24cf67-329d-49c6-ba9f-fdff1e59c267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:53.054034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9a24cf67-329d-49c6-ba9f-fdff1e59c267 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:53.059563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9a24cf67-329d-49c6-ba9f-fdff1e59c267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:53.215825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 898/1795] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:53.226797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:53.228865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:53.277498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:53.335275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e585c6-7684-4a91-8e6f-618389adfcb6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:53.342384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258751b-5840-49ed-8b41-5f7b293d43e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:53.343179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258751b-5840-49ed-8b41-5f7b293d43e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:53.345089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258751b-5840-49ed-8b41-5f7b293d43e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:53.345580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258751b-5840-49ed-8b41-5f7b293d43e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:53.357059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258751b-5840-49ed-8b41-5f7b293d43e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:53.359218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258751b-5840-49ed-8b41-5f7b293d43e5 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:53.376669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 898/1796] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:53.749094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24cf67-329d-49c6-ba9f-fdff1e59c267 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:53.750479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9a24cf67-329d-49c6-ba9f-fdff1e59c267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:53.765207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a24cf67-329d-49c6-ba9f-fdff1e59c267 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:53.771457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:53.772765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:53.773228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:53.773731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] PATH_INFO: `/v3/projects/45e8cae4b6304e5dbf72e13d6608403e` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:53.775287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e8cae4b6304e5dbf72e13d6608403e)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:53.777760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:53.904976 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:53.917016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e8cae4b6304e5dbf72e13d6608403e', 'request_id': 'req-c6a6af8a-67b7-4839-8573-1d2cabb065ee'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:53.928550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e8cae4b6304e5dbf72e13d6608403e', 'request_id': 'req-c6a6af8a-67b7-4839-8573-1d2cabb065ee'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:53.948959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e8cae4b6304e5dbf72e13d6608403e', 'request_id': 'req-c6a6af8a-67b7-4839-8573-1d2cabb065ee'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:53.964114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] The token cache is being invalidate because project 45e8cae4b6304e5dbf72e13d6608403e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:53.965299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:53.966505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c6a6af8a-67b7-4839-8573-1d2cabb065ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:53.968747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 899/1797] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:53 2022] DELETE /identity/v3/projects/45e8cae4b6304e5dbf72e13d6608403e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:53.974147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19442d2f-1403-45cb-9fe4-edff9b312f0d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:53.974661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19442d2f-1403-45cb-9fe4-edff9b312f0d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:53.974782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19442d2f-1403-45cb-9fe4-edff9b312f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:53.999204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-19442d2f-1403-45cb-9fe4-edff9b312f0d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:54.000642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-19442d2f-1403-45cb-9fe4-edff9b312f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:54.095681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 899/1798] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:54.103929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39dfe0-6f7e-4729-8ee1-338969e52222 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.104411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39dfe0-6f7e-4729-8ee1-338969e52222 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.105705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39dfe0-6f7e-4729-8ee1-338969e52222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.137838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-de39dfe0-6f7e-4729-8ee1-338969e52222 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:54.140026 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-de39dfe0-6f7e-4729-8ee1-338969e52222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:54.153167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 900/1799] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:54.155177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19442d2f-1403-45cb-9fe4-edff9b312f0d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:54.177316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19442d2f-1403-45cb-9fe4-edff9b312f0d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:54.187552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:54.190164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.190914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.191613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.193635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:54.195083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:54.200339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:54.227270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39dfe0-6f7e-4729-8ee1-338969e52222 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:54.233921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 900/1800] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:54.242629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39dfe0-6f7e-4729-8ee1-338969e52222 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:54.248714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd3c49-192c-4a68-810a-317afd437be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:54.249559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd3c49-192c-4a68-810a-317afd437be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.250021 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd3c49-192c-4a68-810a-317afd437be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.250437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd3c49-192c-4a68-810a-317afd437be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.251319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd3c49-192c-4a68-810a-317afd437be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:54.253554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd3c49-192c-4a68-810a-317afd437be1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:54.295361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:54.306013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 901/1801] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:54 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:54.312084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651966-f0ab-4df1-bc81-178d8cacdc9a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:54.416250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358ff31-fa8b-492e-93cf-5f603858c113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:54.416250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ff31-fa8b-492e-93cf-5f603858c113 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.416250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ff31-fa8b-492e-93cf-5f603858c113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.416250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358ff31-fa8b-492e-93cf-5f603858c113 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.417185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358ff31-fa8b-492e-93cf-5f603858c113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:54.417185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358ff31-fa8b-492e-93cf-5f603858c113 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:54.457977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 901/1802] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:54 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:54.461913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd3c49-192c-4a68-810a-317afd437be1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:54.489600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd3c49-192c-4a68-810a-317afd437be1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:54.496459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4d4f8-e55d-4ca7-96b9-23a71f294b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:54.497149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4d4f8-e55d-4ca7-96b9-23a71f294b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.497766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4d4f8-e55d-4ca7-96b9-23a71f294b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.498052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4d4f8-e55d-4ca7-96b9-23a71f294b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.498800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4d4f8-e55d-4ca7-96b9-23a71f294b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:54.501065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4d4f8-e55d-4ca7-96b9-23a71f294b0b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:54.529550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358ff31-fa8b-492e-93cf-5f603858c113 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:54.534749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 902/1803] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:54.548027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358ff31-fa8b-492e-93cf-5f603858c113 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:54.555882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61169a44-0f66-4b58-8070-4df951538f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:54.556594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169a44-0f66-4b58-8070-4df951538f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.556878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169a44-0f66-4b58-8070-4df951538f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.557330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61169a44-0f66-4b58-8070-4df951538f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.558712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61169a44-0f66-4b58-8070-4df951538f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:54.560866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61169a44-0f66-4b58-8070-4df951538f8a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:54.581293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 902/1804] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:54.584315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4d4f8-e55d-4ca7-96b9-23a71f294b0b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:54.643702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4d4f8-e55d-4ca7-96b9-23a71f294b0b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:54.650871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a971f9-f2df-4a1e-8bd7-6bd99367e35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:54.665772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a971f9-f2df-4a1e-8bd7-6bd99367e35d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.674983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a971f9-f2df-4a1e-8bd7-6bd99367e35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.674983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a971f9-f2df-4a1e-8bd7-6bd99367e35d admin admin] PATH_INFO: `/v3/projects/87a7c5decdc24b95898519bbf60d77ac/users/7f20a0f9e2d94e1987621b56211519bf/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.676551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cad7a-b1a3-4ce3-8de4-19143cff7ec4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.676864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cad7a-b1a3-4ce3-8de4-19143cff7ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.677150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cad7a-b1a3-4ce3-8de4-19143cff7ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.703294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a971f9-f2df-4a1e-8bd7-6bd99367e35d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a7c5decdc24b95898519bbf60d77ac, user_id=7f20a0f9e2d94e1987621b56211519bf, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:54.708582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a971f9-f2df-4a1e-8bd7-6bd99367e35d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:54.742183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 903/1805] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:54 2022] PUT /identity/v3/projects/87a7c5decdc24b95898519bbf60d77ac/users/7f20a0f9e2d94e1987621b56211519bf/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:54.766337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559017de-5c77-41d2-9b46-b9a5bc1990e9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:54.766822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559017de-5c77-41d2-9b46-b9a5bc1990e9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:54.767241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559017de-5c77-41d2-9b46-b9a5bc1990e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:54.767444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3e5cad7a-b1a3-4ce3-8de4-19143cff7ec4 None None] MFA Rules not processed for user `c26e061da2844459990e1f0a79869468`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:54.783710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3e5cad7a-b1a3-4ce3-8de4-19143cff7ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:54.823955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-559017de-5c77-41d2-9b46-b9a5bc1990e9 None None] MFA Rules not processed for user `7f20a0f9e2d94e1987621b56211519bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:54.827292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-559017de-5c77-41d2-9b46-b9a5bc1990e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:54.835704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:54.921327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 903/1806] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:54.928033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cad7a-b1a3-4ce3-8de4-19143cff7ec4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:54.930854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3e5cad7a-b1a3-4ce3-8de4-19143cff7ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:54.935655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 904/1807] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:55.028504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cad7a-b1a3-4ce3-8de4-19143cff7ec4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:55.041043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:55.041800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.042072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.042990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.043104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:55.045140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:55.077985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 904/1808] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:54 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:55.082635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559017de-5c77-41d2-9b46-b9a5bc1990e9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:55.097124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559017de-5c77-41d2-9b46-b9a5bc1990e9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:55.114365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:55.117020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:55.170938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:55.179931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad334660-913a-41e8-b9e8-4bef10f978f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:55.180651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad334660-913a-41e8-b9e8-4bef10f978f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.181113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad334660-913a-41e8-b9e8-4bef10f978f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.181476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad334660-913a-41e8-b9e8-4bef10f978f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.182258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad334660-913a-41e8-b9e8-4bef10f978f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:55.185713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad334660-913a-41e8-b9e8-4bef10f978f1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:55.210991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 905/1809] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:55 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:55.215053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad334660-913a-41e8-b9e8-4bef10f978f1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:55.221140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f7da0-e2e7-4373-b2c2-41b4d5cc76b6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:55.227072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad334660-913a-41e8-b9e8-4bef10f978f1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:55.229174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74755fd-b22d-4e4d-8ac0-409abe71dc5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:55.229902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74755fd-b22d-4e4d-8ac0-409abe71dc5e service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.230184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74755fd-b22d-4e4d-8ac0-409abe71dc5e service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.230458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74755fd-b22d-4e4d-8ac0-409abe71dc5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.232287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e7a20-5be5-446e-9bcb-a5a6c7bc6b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:55.233015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e7a20-5be5-446e-9bcb-a5a6c7bc6b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.233294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e7a20-5be5-446e-9bcb-a5a6c7bc6b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.233589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e7a20-5be5-446e-9bcb-a5a6c7bc6b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.234344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e7a20-5be5-446e-9bcb-a5a6c7bc6b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:55.236002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e7a20-5be5-446e-9bcb-a5a6c7bc6b6b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:55.250728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 906/1810] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:55.251839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e7a20-5be5-446e-9bcb-a5a6c7bc6b6b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:55.263147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e7a20-5be5-446e-9bcb-a5a6c7bc6b6b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:55.267584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311e7c3-8bfb-422c-9dd0-15114dea44dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:55.268699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311e7c3-8bfb-422c-9dd0-15114dea44dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.268984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311e7c3-8bfb-422c-9dd0-15114dea44dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.269298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311e7c3-8bfb-422c-9dd0-15114dea44dd admin admin] PATH_INFO: `/v3/projects/70fd0f5ab218464eb1a9517b8b4b28b2/users/753a861b73bf4a06b45479bcbd0ae496/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.288490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311e7c3-8bfb-422c-9dd0-15114dea44dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fd0f5ab218464eb1a9517b8b4b28b2, user_id=753a861b73bf4a06b45479bcbd0ae496, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:55.291132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:55.291350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311e7c3-8bfb-422c-9dd0-15114dea44dd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:55.306700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74755fd-b22d-4e4d-8ac0-409abe71dc5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:55.312093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 907/1811] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:55 2022] PUT /identity/v3/projects/70fd0f5ab218464eb1a9517b8b4b28b2/users/753a861b73bf4a06b45479bcbd0ae496/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:55.317590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74755fd-b22d-4e4d-8ac0-409abe71dc5e service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:55.318368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311e7c3-8bfb-422c-9dd0-15114dea44dd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:55.329154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 905/1812] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:55.399717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311e7c3-8bfb-422c-9dd0-15114dea44dd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:55.406164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70866e0-8ddb-4bfa-85be-384c49ce94fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:55.406326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70866e0-8ddb-4bfa-85be-384c49ce94fb admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.406526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70866e0-8ddb-4bfa-85be-384c49ce94fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.406806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70866e0-8ddb-4bfa-85be-384c49ce94fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.407577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70866e0-8ddb-4bfa-85be-384c49ce94fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:55.409587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70866e0-8ddb-4bfa-85be-384c49ce94fb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:55.422897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 908/1813] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:55 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:55.428372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74755fd-b22d-4e4d-8ac0-409abe71dc5e service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:55.442368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74755fd-b22d-4e4d-8ac0-409abe71dc5e service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:55.449730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df46a3-8339-4c31-bd84-345472b23069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:55.449730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df46a3-8339-4c31-bd84-345472b23069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.450556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df46a3-8339-4c31-bd84-345472b23069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.451018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df46a3-8339-4c31-bd84-345472b23069 admin admin] PATH_INFO: `/v3/projects/70fd0f5ab218464eb1a9517b8b4b28b2/users/753a861b73bf4a06b45479bcbd0ae496/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.453512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df46a3-8339-4c31-bd84-345472b23069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70fd0f5ab218464eb1a9517b8b4b28b2, user_id=753a861b73bf4a06b45479bcbd0ae496, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:55.456822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df46a3-8339-4c31-bd84-345472b23069 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:55.478603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 906/1814] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:55 2022] PUT /identity/v3/projects/70fd0f5ab218464eb1a9517b8b4b28b2/users/753a861b73bf4a06b45479bcbd0ae496/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:55.627022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70866e0-8ddb-4bfa-85be-384c49ce94fb admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:55.642176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70866e0-8ddb-4bfa-85be-384c49ce94fb admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:55.767672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:55.769623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.770061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.770436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] PATH_INFO: `/v3/users/7f20a0f9e2d94e1987621b56211519bf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.773730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f20a0f9e2d94e1987621b56211519bf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:55.776214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:55.899932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f20a0f9e2d94e1987621b56211519bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:55.905156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:55.913615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f20a0f9e2d94e1987621b56211519bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:55.959831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9687-2eee-45b3-b082-1654f1477122 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.960134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9687-2eee-45b3-b082-1654f1477122 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.960468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c9687-2eee-45b3-b082-1654f1477122 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:55.963704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 907/1815] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:19:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:19:55.966734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f20a0f9e2d94e1987621b56211519bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:55.968846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f20a0f9e2d94e1987621b56211519bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:55.969834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0357b4c-27c2-4472-bc9c-fddc7f569c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f20a0f9e2d94e1987621b56211519bf'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:55.984317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e2c2e-9b44-4002-b928-51eb3cfb1d4a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:55.984821 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e2c2e-9b44-4002-b928-51eb3cfb1d4a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:55.989419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e2c2e-9b44-4002-b928-51eb3cfb1d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:56.019846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 909/1816] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:55 2022] DELETE /identity/v3/users/7f20a0f9e2d94e1987621b56211519bf => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:56.022883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f63ed9-2171-4370-b2e8-49de227b9b8c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:56.023294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f63ed9-2171-4370-b2e8-49de227b9b8c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:56.023683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f63ed9-2171-4370-b2e8-49de227b9b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:56.027166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0e8e2c2e-9b44-4002-b928-51eb3cfb1d4a None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:56.029153 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0e8e2c2e-9b44-4002-b928-51eb3cfb1d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:56.066413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-70f63ed9-2171-4370-b2e8-49de227b9b8c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:56.067857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-70f63ed9-2171-4370-b2e8-49de227b9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:56.141933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 908/1817] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:19:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:56.171280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 910/1818] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:56.182185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e2c2e-9b44-4002-b928-51eb3cfb1d4a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:56.184046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0e8e2c2e-9b44-4002-b928-51eb3cfb1d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:56.257772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e2c2e-9b44-4002-b928-51eb3cfb1d4a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:56.261707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930c3c3-5075-4205-b5bb-6120cce56873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:56.262440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930c3c3-5075-4205-b5bb-6120cce56873 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:56.262792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930c3c3-5075-4205-b5bb-6120cce56873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:56.263188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930c3c3-5075-4205-b5bb-6120cce56873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:56.276148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930c3c3-5075-4205-b5bb-6120cce56873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:56.278498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930c3c3-5075-4205-b5bb-6120cce56873 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:56.301199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 909/1819] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:56.494528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f63ed9-2171-4370-b2e8-49de227b9b8c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:56.496453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-70f63ed9-2171-4370-b2e8-49de227b9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:56.517116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f63ed9-2171-4370-b2e8-49de227b9b8c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:56.523798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:56.524484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:56.524891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:56.525110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:56.535890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:56.537515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:56.551020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 911/1820] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:56.913559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525fbf-eb20-456a-bbb7-75a4a288e6a7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:56.913946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525fbf-eb20-456a-bbb7-75a4a288e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:56.914207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525fbf-eb20-456a-bbb7-75a4a288e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:56.951742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-73525fbf-eb20-456a-bbb7-75a4a288e6a7 None None] MFA Rules not processed for user `753a861b73bf4a06b45479bcbd0ae496`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:56.954309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-73525fbf-eb20-456a-bbb7-75a4a288e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:56.985624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:56.988290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.011941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d4d3e-d1b7-48e7-bde5-744d169e05ba service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.019674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.020665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.020947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.021276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] PATH_INFO: `/v3/projects/87a7c5decdc24b95898519bbf60d77ac` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.022592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a7c5decdc24b95898519bbf60d77ac)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.025383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.054203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:57.067568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 910/1821] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:57.086774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525fbf-eb20-456a-bbb7-75a4a288e6a7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.089644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-73525fbf-eb20-456a-bbb7-75a4a288e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.106679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a7c5decdc24b95898519bbf60d77ac', 'request_id': 'req-ecee1b78-07f8-4200-975b-e9b429cfc465'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.124290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a7c5decdc24b95898519bbf60d77ac', 'request_id': 'req-ecee1b78-07f8-4200-975b-e9b429cfc465'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.151533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a7c5decdc24b95898519bbf60d77ac', 'request_id': 'req-ecee1b78-07f8-4200-975b-e9b429cfc465'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.169790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] The token cache is being invalidate because project 87a7c5decdc24b95898519bbf60d77ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:19:57.170221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.171395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ecee1b78-07f8-4200-975b-e9b429cfc465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:57.175041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 912/1822] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:56 2022] DELETE /identity/v3/projects/87a7c5decdc24b95898519bbf60d77ac => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:19:57.181998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb844746-7862-415b-a82f-26adcb16edeb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.182783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb844746-7862-415b-a82f-26adcb16edeb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.182955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525fbf-eb20-456a-bbb7-75a4a288e6a7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.184372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb844746-7862-415b-a82f-26adcb16edeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.207623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-eb844746-7862-415b-a82f-26adcb16edeb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:57.209172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-eb844746-7862-415b-a82f-26adcb16edeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.249344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:57.250736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.251451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.251792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.252005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.254196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.289458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 913/1823] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:19:57.300010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb366d-f04b-4f64-b4b5-fc0af916bc29 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.300390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb366d-f04b-4f64-b4b5-fc0af916bc29 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.300810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb366d-f04b-4f64-b4b5-fc0af916bc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.324519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f9bb366d-f04b-4f64-b4b5-fc0af916bc29 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:57.325796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f9bb366d-f04b-4f64-b4b5-fc0af916bc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.334470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 914/1824] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:57.338531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb366d-f04b-4f64-b4b5-fc0af916bc29 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.340392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.341905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.350811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb366d-f04b-4f64-b4b5-fc0af916bc29 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.352199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 911/1825] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:57.356671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.358105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.358602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.359382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.360485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:19:57.361252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.363768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.378450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 915/1826] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:19:57.382608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.394643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a02b79-55d3-48ec-b183-a8841a452fce service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.399685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff27de-d4e4-4d0b-993f-4019071e0f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.400408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff27de-d4e4-4d0b-993f-4019071e0f76 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.400726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff27de-d4e4-4d0b-993f-4019071e0f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.401009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff27de-d4e4-4d0b-993f-4019071e0f76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.401787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff27de-d4e4-4d0b-993f-4019071e0f76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.403869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff27de-d4e4-4d0b-993f-4019071e0f76 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.436130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 912/1827] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:57.443547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.454650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b22c7-c339-47a0-8648-7fb2b3bfbdce admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.527462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596ea07-b39c-4b86-92b1-fe4451deff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.528257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ea07-b39c-4b86-92b1-fe4451deff41 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.528604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ea07-b39c-4b86-92b1-fe4451deff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.528946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ea07-b39c-4b86-92b1-fe4451deff41 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.529798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596ea07-b39c-4b86-92b1-fe4451deff41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.532261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596ea07-b39c-4b86-92b1-fe4451deff41 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.568317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 916/1828] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:19:57.573732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff27de-d4e4-4d0b-993f-4019071e0f76 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.589994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff27de-d4e4-4d0b-993f-4019071e0f76 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.597393 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc31e78-615f-48fb-9cdd-73d90bb709b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.598234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc31e78-615f-48fb-9cdd-73d90bb709b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.598594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc31e78-615f-48fb-9cdd-73d90bb709b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.598934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc31e78-615f-48fb-9cdd-73d90bb709b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.599784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc31e78-615f-48fb-9cdd-73d90bb709b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.601379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc31e78-615f-48fb-9cdd-73d90bb709b9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.618591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 913/1829] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:57.621618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596ea07-b39c-4b86-92b1-fe4451deff41 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.638867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596ea07-b39c-4b86-92b1-fe4451deff41 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.646361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846735e4-c01b-4082-ae8e-642af1795788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.647164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846735e4-c01b-4082-ae8e-642af1795788 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.647549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846735e4-c01b-4082-ae8e-642af1795788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.647933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846735e4-c01b-4082-ae8e-642af1795788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.648813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846735e4-c01b-4082-ae8e-642af1795788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.651427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846735e4-c01b-4082-ae8e-642af1795788 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.663188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 917/1830] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:57.669210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc31e78-615f-48fb-9cdd-73d90bb709b9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.685300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc31e78-615f-48fb-9cdd-73d90bb709b9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.705255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9973a2e-06a7-42c3-8a44-6be78da09cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.706240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9973a2e-06a7-42c3-8a44-6be78da09cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.706529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9973a2e-06a7-42c3-8a44-6be78da09cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.706781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9973a2e-06a7-42c3-8a44-6be78da09cdc admin admin] PATH_INFO: `/v3/projects/a654ddc3af1942f3b7539d4bf0865c2f/users/a8e0d6a1ee6047b784b454dd5057b8d2/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.728406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9973a2e-06a7-42c3-8a44-6be78da09cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a654ddc3af1942f3b7539d4bf0865c2f, user_id=a8e0d6a1ee6047b784b454dd5057b8d2, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.731675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9973a2e-06a7-42c3-8a44-6be78da09cdc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:57.758653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 914/1831] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:57 2022] PUT /identity/v3/projects/a654ddc3af1942f3b7539d4bf0865c2f/users/a8e0d6a1ee6047b784b454dd5057b8d2/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:57.765245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11932-209c-4c61-bce8-b8e792126652 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.765897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11932-209c-4c61-bce8-b8e792126652 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.766423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a11932-209c-4c61-bce8-b8e792126652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.798773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-06a11932-209c-4c61-bce8-b8e792126652 None None] MFA Rules not processed for user `a8e0d6a1ee6047b784b454dd5057b8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:57.800119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-06a11932-209c-4c61-bce8-b8e792126652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:57.885578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 918/1832] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:57.890839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9973a2e-06a7-42c3-8a44-6be78da09cdc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:57.901587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:57.962216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9973a2e-06a7-42c3-8a44-6be78da09cdc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:57.967350 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4ed2e-4eb1-4d21-bff8-04a70bc81f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:57.968009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4ed2e-4eb1-4d21-bff8-04a70bc81f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:57.968296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4ed2e-4eb1-4d21-bff8-04a70bc81f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:57.968577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4ed2e-4eb1-4d21-bff8-04a70bc81f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:57.969309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4ed2e-4eb1-4d21-bff8-04a70bc81f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:57.971531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4ed2e-4eb1-4d21-bff8-04a70bc81f77 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.005051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 915/1833] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:19:57 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:58.009722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a11932-209c-4c61-bce8-b8e792126652 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.022724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a11932-209c-4c61-bce8-b8e792126652 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.083187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:58.122554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da10349-2cee-4764-bfe2-76fe75944b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.123644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da10349-2cee-4764-bfe2-76fe75944b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.123841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da10349-2cee-4764-bfe2-76fe75944b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.124118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da10349-2cee-4764-bfe2-76fe75944b2c admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.124876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da10349-2cee-4764-bfe2-76fe75944b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.127499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da10349-2cee-4764-bfe2-76fe75944b2c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.150915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 919/1834] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:19:58 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:19:58.154763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4ed2e-4eb1-4d21-bff8-04a70bc81f77 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.167414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4ed2e-4eb1-4d21-bff8-04a70bc81f77 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.172368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb075b68-eb26-42bc-bda0-3bc897584c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.173052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075b68-eb26-42bc-bda0-3bc897584c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.173338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075b68-eb26-42bc-bda0-3bc897584c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.173586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb075b68-eb26-42bc-bda0-3bc897584c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.174889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb075b68-eb26-42bc-bda0-3bc897584c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.176474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb075b68-eb26-42bc-bda0-3bc897584c15 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.189448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 916/1835] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:19:58.194487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da10349-2cee-4764-bfe2-76fe75944b2c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.206020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da10349-2cee-4764-bfe2-76fe75944b2c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.211425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a184b88-b24b-4729-b697-c34dfe591a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.212335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a184b88-b24b-4729-b697-c34dfe591a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.212620 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a184b88-b24b-4729-b697-c34dfe591a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.212898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a184b88-b24b-4729-b697-c34dfe591a07 admin admin] PATH_INFO: `/v3/projects/c781a6b989f14284b219ac0603149c43/users/d1d1d41062044be580d98440b02517bc/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.230887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a184b88-b24b-4729-b697-c34dfe591a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c781a6b989f14284b219ac0603149c43, user_id=d1d1d41062044be580d98440b02517bc, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.233933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a184b88-b24b-4729-b697-c34dfe591a07 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.249771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 920/1836] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:58 2022] PUT /identity/v3/projects/c781a6b989f14284b219ac0603149c43/users/d1d1d41062044be580d98440b02517bc/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:58.254211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb075b68-eb26-42bc-bda0-3bc897584c15 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.265045 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb075b68-eb26-42bc-bda0-3bc897584c15 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.334399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b26a8-f9fc-41a9-8e1b-87d4584287d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.335149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b26a8-f9fc-41a9-8e1b-87d4584287d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.335450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b26a8-f9fc-41a9-8e1b-87d4584287d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.335731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b26a8-f9fc-41a9-8e1b-87d4584287d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.336470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b26a8-f9fc-41a9-8e1b-87d4584287d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.338360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b26a8-f9fc-41a9-8e1b-87d4584287d3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.349529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 917/1837] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:19:58.358248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a184b88-b24b-4729-b697-c34dfe591a07 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.371349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a184b88-b24b-4729-b697-c34dfe591a07 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.377322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.378432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.379348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.379348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] PATH_INFO: `/v3/projects/c781a6b989f14284b219ac0603149c43/users/d1d1d41062044be580d98440b02517bc/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.381055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c781a6b989f14284b219ac0603149c43, user_id=d1d1d41062044be580d98440b02517bc, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.383849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.397234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 921/1838] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:19:58 2022] PUT /identity/v3/projects/c781a6b989f14284b219ac0603149c43/users/d1d1d41062044be580d98440b02517bc/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:19:58.407517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5704d7d-f877-4122-8fb8-2735976dd0bb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.407912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5704d7d-f877-4122-8fb8-2735976dd0bb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.408132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5704d7d-f877-4122-8fb8-2735976dd0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.442942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e5704d7d-f877-4122-8fb8-2735976dd0bb None None] MFA Rules not processed for user `d1d1d41062044be580d98440b02517bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:58.444273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e5704d7d-f877-4122-8fb8-2735976dd0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:58.537119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 918/1839] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:58.550836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:58.554124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:58.628176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938749d-13b6-4440-a129-7fc67396c2c0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:58.638209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3dc91-207f-430e-9639-01edb4105eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:58.638892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3dc91-207f-430e-9639-01edb4105eac service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:58.639178 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3dc91-207f-430e-9639-01edb4105eac service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:58.639455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3dc91-207f-430e-9639-01edb4105eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:58.650547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3dc91-207f-430e-9639-01edb4105eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:58.652016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3dc91-207f-430e-9639-01edb4105eac service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:58.662970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 922/1840] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:58 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:59.116661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5704d7d-f877-4122-8fb8-2735976dd0bb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:59.122651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:59.189765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5704d7d-f877-4122-8fb8-2735976dd0bb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:59.195669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:59.196545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:59.196829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:59.197107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] PATH_INFO: `/v3/users/a8e0d6a1ee6047b784b454dd5057b8d2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:59.198453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e0d6a1ee6047b784b454dd5057b8d2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:59.200493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:59.260889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e0d6a1ee6047b784b454dd5057b8d2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:59.272038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e0d6a1ee6047b784b454dd5057b8d2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:59.295618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e0d6a1ee6047b784b454dd5057b8d2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:59.296342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e0d6a1ee6047b784b454dd5057b8d2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:59.297172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e0d6a1ee6047b784b454dd5057b8d2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:19:59.316212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 919/1841] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:19:59 2022] DELETE /identity/v3/users/a8e0d6a1ee6047b784b454dd5057b8d2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:19:59.326256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eabf6-af93-4e73-868e-3e1661f3e486 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:59.327057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eabf6-af93-4e73-868e-3e1661f3e486 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:59.327221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eabf6-af93-4e73-868e-3e1661f3e486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:59.348980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-124eabf6-af93-4e73-868e-3e1661f3e486 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:19:59.350358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-124eabf6-af93-4e73-868e-3e1661f3e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:59.420354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 923/1842] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:19:59.430901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:59.432417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:59.499656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddeff5c-ae1d-40f3-8d46-cc30e864d5c4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:59.504186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:59.504864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:59.505147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:59.505423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:59.528608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:59.530155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:59.552088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 920/1843] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:19:59.896765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eabf6-af93-4e73-868e-3e1661f3e486 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:59.898759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-124eabf6-af93-4e73-868e-3e1661f3e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:59.911418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:19:59.912955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:19:59.927976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27a37e-4d43-438c-883b-8ac8c8ce22f7 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:19:59.943325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:19:59.944258 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:19:59.944543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:19:59.944859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] PATH_INFO: `/v3/projects/a654ddc3af1942f3b7539d4bf0865c2f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:19:59.946730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a654ddc3af1942f3b7539d4bf0865c2f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:19:59.947872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:59.949237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:19:59.979332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:19:59.997333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eabf6-af93-4e73-868e-3e1661f3e486 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:00.010322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a654ddc3af1942f3b7539d4bf0865c2f', 'request_id': 'req-5ca730b3-b6df-49a5-adee-22a11febdb02'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.021895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a654ddc3af1942f3b7539d4bf0865c2f', 'request_id': 'req-5ca730b3-b6df-49a5-adee-22a11febdb02'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.030783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a654ddc3af1942f3b7539d4bf0865c2f', 'request_id': 'req-5ca730b3-b6df-49a5-adee-22a11febdb02'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.045745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] The token cache is being invalidate because project a654ddc3af1942f3b7539d4bf0865c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:00.046179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.046958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.049364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 921/1844] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:19:59 2022] DELETE /identity/v3/projects/a654ddc3af1942f3b7539d4bf0865c2f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:00.053318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:00.054852 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:00.055993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:00.056894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:00.057209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:00.057476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:00.059900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:00.147735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:00.149257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:00.160747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 924/1845] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:19:59 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:00.169498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca730b3-b6df-49a5-adee-22a11febdb02 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:00.175198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:00.176109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:00.176386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:00.176660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] PATH_INFO: `/v3/users/d1d1d41062044be580d98440b02517bc` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:00.177928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d1d41062044be580d98440b02517bc)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:00.179960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:00.237368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d1d41062044be580d98440b02517bc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.247222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d1d41062044be580d98440b02517bc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.264499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d1d41062044be580d98440b02517bc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.265352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d1d41062044be580d98440b02517bc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.266069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d168fd38-c700-4042-a340-ba54ba5ff415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d1d41062044be580d98440b02517bc'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.280966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 922/1846] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:00 2022] DELETE /identity/v3/users/d1d1d41062044be580d98440b02517bc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:00.637332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:00.651960 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9953660-15e9-46d2-8ec0-2ddb05f6fbc4 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:00.716709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:00.717591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:00.717908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:00.718182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] PATH_INFO: `/v3/projects/c781a6b989f14284b219ac0603149c43` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:00.719400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] RBAC: Authorizing `identity:delete_project(project_id=c781a6b989f14284b219ac0603149c43)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:00.721557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:00.764538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:00.782231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c781a6b989f14284b219ac0603149c43', 'request_id': 'req-79796b71-45b3-44f6-a0f3-e5c11f358cad'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.792968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c781a6b989f14284b219ac0603149c43', 'request_id': 'req-79796b71-45b3-44f6-a0f3-e5c11f358cad'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.804154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c781a6b989f14284b219ac0603149c43', 'request_id': 'req-79796b71-45b3-44f6-a0f3-e5c11f358cad'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.819816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] The token cache is being invalidate because project c781a6b989f14284b219ac0603149c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:00.820112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.820909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79796b71-45b3-44f6-a0f3-e5c11f358cad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:00.822652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 925/1847] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:00 2022] DELETE /identity/v3/projects/c781a6b989f14284b219ac0603149c43 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:00.828434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04239e1-b8ba-4ec1-80f7-5534968fb803 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:00.828750 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04239e1-b8ba-4ec1-80f7-5534968fb803 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:00.829033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04239e1-b8ba-4ec1-80f7-5534968fb803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:00.851569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a04239e1-b8ba-4ec1-80f7-5534968fb803 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:00.852321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a04239e1-b8ba-4ec1-80f7-5534968fb803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:00.884539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:00.929619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 923/1848] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:00.938874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765ae5-f2e1-499a-9dde-1f3b0e5b6639 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:00.939263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765ae5-f2e1-499a-9dde-1f3b0e5b6639 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:00.939708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765ae5-f2e1-499a-9dde-1f3b0e5b6639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:00.959342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-80765ae5-f2e1-499a-9dde-1f3b0e5b6639 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:00.960560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-80765ae5-f2e1-499a-9dde-1f3b0e5b6639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:00.969237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 926/1849] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:00.973977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04239e1-b8ba-4ec1-80f7-5534968fb803 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:00.983907 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04239e1-b8ba-4ec1-80f7-5534968fb803 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:00.989711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:00.990543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:00.990827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:00.991103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:00.992086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:00.992404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:00.994082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.009930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 924/1850] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:01.011073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80765ae5-f2e1-499a-9dde-1f3b0e5b6639 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.021047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80765ae5-f2e1-499a-9dde-1f3b0e5b6639 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.025779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed0035-684e-46e5-b8a2-3ca9f2ea660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.026428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed0035-684e-46e5-b8a2-3ca9f2ea660c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.026706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed0035-684e-46e5-b8a2-3ca9f2ea660c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.026983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed0035-684e-46e5-b8a2-3ca9f2ea660c admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.027672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed0035-684e-46e5-b8a2-3ca9f2ea660c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.029588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed0035-684e-46e5-b8a2-3ca9f2ea660c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.056899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 927/1851] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:01 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:01.061419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.072065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65a69f-2c1d-4187-b5b1-4c61513cb4e5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.141838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837439fb-190e-4953-a0ee-eaeaf1301cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.142897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837439fb-190e-4953-a0ee-eaeaf1301cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.143710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837439fb-190e-4953-a0ee-eaeaf1301cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.143982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837439fb-190e-4953-a0ee-eaeaf1301cc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.144718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837439fb-190e-4953-a0ee-eaeaf1301cc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.146751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837439fb-190e-4953-a0ee-eaeaf1301cc6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.176478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 925/1852] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:01 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:01.177935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed0035-684e-46e5-b8a2-3ca9f2ea660c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.190729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed0035-684e-46e5-b8a2-3ca9f2ea660c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.196240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95fa1f-b9dd-430d-852a-6403d3bf26bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.196935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95fa1f-b9dd-430d-852a-6403d3bf26bc admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.197213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95fa1f-b9dd-430d-852a-6403d3bf26bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.197486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95fa1f-b9dd-430d-852a-6403d3bf26bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.198245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95fa1f-b9dd-430d-852a-6403d3bf26bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.199879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95fa1f-b9dd-430d-852a-6403d3bf26bc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.212318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 928/1853] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:01.217044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837439fb-190e-4953-a0ee-eaeaf1301cc6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.233893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837439fb-190e-4953-a0ee-eaeaf1301cc6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.239686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6c952-a16f-4dac-b0d1-4606134de129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.240379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6c952-a16f-4dac-b0d1-4606134de129 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.240646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6c952-a16f-4dac-b0d1-4606134de129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.240919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6c952-a16f-4dac-b0d1-4606134de129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.241795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6c952-a16f-4dac-b0d1-4606134de129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.243457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6c952-a16f-4dac-b0d1-4606134de129 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.255103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 926/1854] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:01.259366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95fa1f-b9dd-430d-852a-6403d3bf26bc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.270844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95fa1f-b9dd-430d-852a-6403d3bf26bc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.276242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730e01-33ac-4513-945a-482bcc9eae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.277221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730e01-33ac-4513-945a-482bcc9eae98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.277521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730e01-33ac-4513-945a-482bcc9eae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.277844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84730e01-33ac-4513-945a-482bcc9eae98 admin admin] PATH_INFO: `/v3/projects/6d5011276c1f49f5bd18a1a9cacd8b5f/users/3c00cda325b54c6b830df582295c35aa/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.296858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84730e01-33ac-4513-945a-482bcc9eae98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d5011276c1f49f5bd18a1a9cacd8b5f, user_id=3c00cda325b54c6b830df582295c35aa, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.300967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84730e01-33ac-4513-945a-482bcc9eae98 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.319092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 929/1855] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:01 2022] PUT /identity/v3/projects/6d5011276c1f49f5bd18a1a9cacd8b5f/users/3c00cda325b54c6b830df582295c35aa/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:01.329431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e4d10-e710-42c0-b9b1-ce5c9ce45bf7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.329761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e4d10-e710-42c0-b9b1-ce5c9ce45bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.330098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e4d10-e710-42c0-b9b1-ce5c9ce45bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.362879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6f6e4d10-e710-42c0-b9b1-ce5c9ce45bf7 None None] MFA Rules not processed for user `3c00cda325b54c6b830df582295c35aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:01.365123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6f6e4d10-e710-42c0-b9b1-ce5c9ce45bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:01.456779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 927/1856] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:01.457864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730e01-33ac-4513-945a-482bcc9eae98 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.540175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84730e01-33ac-4513-945a-482bcc9eae98 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.545163 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd54d1-6347-4757-baf7-1e1f1fcb6552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.545859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd54d1-6347-4757-baf7-1e1f1fcb6552 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.546144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd54d1-6347-4757-baf7-1e1f1fcb6552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.546417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd54d1-6347-4757-baf7-1e1f1fcb6552 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.547171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd54d1-6347-4757-baf7-1e1f1fcb6552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.549175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd54d1-6347-4757-baf7-1e1f1fcb6552 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.578617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 930/1857] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:01 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:01.583204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e4d10-e710-42c0-b9b1-ce5c9ce45bf7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.594944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e4d10-e710-42c0-b9b1-ce5c9ce45bf7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.617163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:01.667758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00d0b0-b74e-44f0-8b5a-f1c8bd5d1ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.668848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00d0b0-b74e-44f0-8b5a-f1c8bd5d1ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.669220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00d0b0-b74e-44f0-8b5a-f1c8bd5d1ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.669707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00d0b0-b74e-44f0-8b5a-f1c8bd5d1ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.670655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00d0b0-b74e-44f0-8b5a-f1c8bd5d1ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.673392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00d0b0-b74e-44f0-8b5a-f1c8bd5d1ca4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.695228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 928/1858] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:01 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:01.699673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd54d1-6347-4757-baf7-1e1f1fcb6552 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.714639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd54d1-6347-4757-baf7-1e1f1fcb6552 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.720453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e89a-8232-4803-bf64-39f445fd7b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.721722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e89a-8232-4803-bf64-39f445fd7b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.722014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e89a-8232-4803-bf64-39f445fd7b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.722293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915e89a-8232-4803-bf64-39f445fd7b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.723042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915e89a-8232-4803-bf64-39f445fd7b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.724658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915e89a-8232-4803-bf64-39f445fd7b91 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.736396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 931/1859] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:01.741253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00d0b0-b74e-44f0-8b5a-f1c8bd5d1ca4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.752527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00d0b0-b74e-44f0-8b5a-f1c8bd5d1ca4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.757666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796c66-ba11-4e3a-a86d-7abcf9e3d46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.758589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796c66-ba11-4e3a-a86d-7abcf9e3d46d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.758871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796c66-ba11-4e3a-a86d-7abcf9e3d46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.759151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796c66-ba11-4e3a-a86d-7abcf9e3d46d admin admin] PATH_INFO: `/v3/projects/961bb4f92f094510b4168595225a351f/users/d84b3cabe9f94e7fb163fb2c19265eb7/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.775099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf796c66-ba11-4e3a-a86d-7abcf9e3d46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=961bb4f92f094510b4168595225a351f, user_id=d84b3cabe9f94e7fb163fb2c19265eb7, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.777868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf796c66-ba11-4e3a-a86d-7abcf9e3d46d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.793410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 929/1860] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:01 2022] PUT /identity/v3/projects/961bb4f92f094510b4168595225a351f/users/d84b3cabe9f94e7fb163fb2c19265eb7/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:01.797687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e89a-8232-4803-bf64-39f445fd7b91 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.811442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915e89a-8232-4803-bf64-39f445fd7b91 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.847067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:01.884436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1eab5-80eb-4f3d-801c-6598f1de27eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.885119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1eab5-80eb-4f3d-801c-6598f1de27eb admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.885400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1eab5-80eb-4f3d-801c-6598f1de27eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.885718 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1eab5-80eb-4f3d-801c-6598f1de27eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.886492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1eab5-80eb-4f3d-801c-6598f1de27eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.888249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1eab5-80eb-4f3d-801c-6598f1de27eb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.899658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 932/1861] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:01.906433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796c66-ba11-4e3a-a86d-7abcf9e3d46d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:01.922781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796c66-ba11-4e3a-a86d-7abcf9e3d46d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:01.929071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:01.930416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.930416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.930803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] PATH_INFO: `/v3/projects/961bb4f92f094510b4168595225a351f/users/d84b3cabe9f94e7fb163fb2c19265eb7/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.933145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=961bb4f92f094510b4168595225a351f, user_id=d84b3cabe9f94e7fb163fb2c19265eb7, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:01.935894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:01.951019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 930/1862] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:01 2022] PUT /identity/v3/projects/961bb4f92f094510b4168595225a351f/users/d84b3cabe9f94e7fb163fb2c19265eb7/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:01.963480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c8395-c9c5-4618-9d99-3b204c163552 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:01.963774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c8395-c9c5-4618-9d99-3b204c163552 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:01.964050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c8395-c9c5-4618-9d99-3b204c163552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:01.994623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-086c8395-c9c5-4618-9d99-3b204c163552 None None] MFA Rules not processed for user `d84b3cabe9f94e7fb163fb2c19265eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:01.995891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-086c8395-c9c5-4618-9d99-3b204c163552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:02.083228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 933/1863] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:01 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:02.096832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:02.099409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:02.167381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d2c3c-d0a3-40e7-b3b3-daf99b80dfa8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:02.171179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eeaea-0daf-460f-8917-aed3d65f907c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:02.171875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eeaea-0daf-460f-8917-aed3d65f907c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:02.172150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eeaea-0daf-460f-8917-aed3d65f907c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:02.172485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eeaea-0daf-460f-8917-aed3d65f907c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:02.183313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eeaea-0daf-460f-8917-aed3d65f907c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:02.184825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eeaea-0daf-460f-8917-aed3d65f907c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:02.195934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 931/1864] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:02.285920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c8395-c9c5-4618-9d99-3b204c163552 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:02.301393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086c8395-c9c5-4618-9d99-3b204c163552 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:02.307356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b740527-77c5-4520-a6bc-5e6a236cdd95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:02.308107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b740527-77c5-4520-a6bc-5e6a236cdd95 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:02.308393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b740527-77c5-4520-a6bc-5e6a236cdd95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:02.308731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b740527-77c5-4520-a6bc-5e6a236cdd95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:02.394005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b740527-77c5-4520-a6bc-5e6a236cdd95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:02.396008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b740527-77c5-4520-a6bc-5e6a236cdd95 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:02.411509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 934/1865] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:05.670010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eeaea-0daf-460f-8917-aed3d65f907c service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:05.744288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eeaea-0daf-460f-8917-aed3d65f907c service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:05.749410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:05.750450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:05.750752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:05.751165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] PATH_INFO: `/v3/users/3c00cda325b54c6b830df582295c35aa` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:05.752558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c00cda325b54c6b830df582295c35aa)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:05.754729 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:05.794494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:05.840785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c00cda325b54c6b830df582295c35aa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:05.851111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c00cda325b54c6b830df582295c35aa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:05.868506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c00cda325b54c6b830df582295c35aa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:05.869330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c00cda325b54c6b830df582295c35aa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:05.870610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c00cda325b54c6b830df582295c35aa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:05.883796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 932/1866] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:05 2022] DELETE /identity/v3/users/3c00cda325b54c6b830df582295c35aa => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:05.888762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aaaa0-24cf-4fba-b011-3f712f11dd9a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:05.889300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aaaa0-24cf-4fba-b011-3f712f11dd9a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:05.889300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aaaa0-24cf-4fba-b011-3f712f11dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:05.913835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9c8aaaa0-24cf-4fba-b011-3f712f11dd9a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:05.916312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9c8aaaa0-24cf-4fba-b011-3f712f11dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:05.966187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:05.983910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 935/1867] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:05.998180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:05.999826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:06.065739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb579c8-2dcc-4858-9ab6-dc4bbe020f01 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:06.070424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eb39a-8d5a-4471-a160-7fca22856ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:06.071258 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eb39a-8d5a-4471-a160-7fca22856ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:06.071546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eb39a-8d5a-4471-a160-7fca22856ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:06.071818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360eb39a-8d5a-4471-a160-7fca22856ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:06.084043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360eb39a-8d5a-4471-a160-7fca22856ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:06.084200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360eb39a-8d5a-4471-a160-7fca22856ae7 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:06.094945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 933/1868] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:06.756055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8aaaa0-24cf-4fba-b011-3f712f11dd9a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:06.758250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9c8aaaa0-24cf-4fba-b011-3f712f11dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:06.774386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8aaaa0-24cf-4fba-b011-3f712f11dd9a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:06.781640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:06.783128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:06.783871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:06.784434 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] PATH_INFO: `/v3/projects/6d5011276c1f49f5bd18a1a9cacd8b5f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:06.786416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d5011276c1f49f5bd18a1a9cacd8b5f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:06.791148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:06.870952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d5011276c1f49f5bd18a1a9cacd8b5f', 'request_id': 'req-4c4e30a6-6099-4636-8b30-805e730c2c21'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:06.883456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d5011276c1f49f5bd18a1a9cacd8b5f', 'request_id': 'req-4c4e30a6-6099-4636-8b30-805e730c2c21'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:06.898207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d5011276c1f49f5bd18a1a9cacd8b5f', 'request_id': 'req-4c4e30a6-6099-4636-8b30-805e730c2c21'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:06.912925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] The token cache is being invalidate because project 6d5011276c1f49f5bd18a1a9cacd8b5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:06.913275 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:06.914211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:06.916389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 936/1869] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:06 2022] DELETE /identity/v3/projects/6d5011276c1f49f5bd18a1a9cacd8b5f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:06.921352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eb39a-8d5a-4471-a160-7fca22856ae7 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:06.933326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360eb39a-8d5a-4471-a160-7fca22856ae7 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:07.013002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:07.014844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:07.015338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:07.015973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] PATH_INFO: `/v3/users/d84b3cabe9f94e7fb163fb2c19265eb7` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:07.018066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d84b3cabe9f94e7fb163fb2c19265eb7)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:07.021174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:07.097595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:07.100160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd84b3cabe9f94e7fb163fb2c19265eb7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.112942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd84b3cabe9f94e7fb163fb2c19265eb7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.126576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd84b3cabe9f94e7fb163fb2c19265eb7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.127583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd84b3cabe9f94e7fb163fb2c19265eb7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.128239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-70fef5dc-8c9b-4359-b986-eb1097f75209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd84b3cabe9f94e7fb163fb2c19265eb7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.141090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 934/1870] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:06 2022] DELETE /identity/v3/users/d84b3cabe9f94e7fb163fb2c19265eb7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:07.516380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:07.518101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:07.539524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e30a6-6099-4636-8b30-805e730c2c21 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:07.619919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:07.620790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:07.621079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:07.621356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] PATH_INFO: `/v3/projects/961bb4f92f094510b4168595225a351f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:07.622624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=961bb4f92f094510b4168595225a351f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:07.624584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:07.706255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '961bb4f92f094510b4168595225a351f', 'request_id': 'req-a7764b22-7bbc-4331-b321-5dca6dd9a020'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.714896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '961bb4f92f094510b4168595225a351f', 'request_id': 'req-a7764b22-7bbc-4331-b321-5dca6dd9a020'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.723716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '961bb4f92f094510b4168595225a351f', 'request_id': 'req-a7764b22-7bbc-4331-b321-5dca6dd9a020'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.737917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] The token cache is being invalidate because project 961bb4f92f094510b4168595225a351f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:07.738298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.739290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a7764b22-7bbc-4331-b321-5dca6dd9a020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:07.745214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 937/1871] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:07 2022] DELETE /identity/v3/projects/961bb4f92f094510b4168595225a351f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:07.746344 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8960-e698-45a5-911e-e4223c33a8b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:07.746641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8960-e698-45a5-911e-e4223c33a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:07.746996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d8960-e698-45a5-911e-e4223c33a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:07.771238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f86d8960-e698-45a5-911e-e4223c33a8b0 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:07.772503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f86d8960-e698-45a5-911e-e4223c33a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:07.872863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 935/1872] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:07.874266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832bf29-dfaf-478c-a577-9134b6727911 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:07.874500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832bf29-dfaf-478c-a577-9134b6727911 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:07.874834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832bf29-dfaf-478c-a577-9134b6727911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:07.883534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:07.899810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c832bf29-dfaf-478c-a577-9134b6727911 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:07.902074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c832bf29-dfaf-478c-a577-9134b6727911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:07.912452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 938/1873] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:07.917146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d8960-e698-45a5-911e-e4223c33a8b0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:07.931231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d8960-e698-45a5-911e-e4223c33a8b0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:07.936737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:07.937438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:07.937752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:07.938030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:07.938793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:07.939095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:07.941873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:07.955074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 936/1874] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:07.959258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832bf29-dfaf-478c-a577-9134b6727911 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:07.970411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832bf29-dfaf-478c-a577-9134b6727911 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:07.976313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3f760-9beb-4028-ac5c-784a77e141b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:07.977037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3f760-9beb-4028-ac5c-784a77e141b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:07.977320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3f760-9beb-4028-ac5c-784a77e141b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:07.977557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3f760-9beb-4028-ac5c-784a77e141b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:07.978510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3f760-9beb-4028-ac5c-784a77e141b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:07.980520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3f760-9beb-4028-ac5c-784a77e141b6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:08.016595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 939/1875] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:07 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:08.020981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:08.033739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de891a3-048c-4e5c-bcd3-286943ca23fa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:08.183132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcff1a1-b8ab-471a-8b59-f40eb77e4c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:08.183934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcff1a1-b8ab-471a-8b59-f40eb77e4c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:08.184291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcff1a1-b8ab-471a-8b59-f40eb77e4c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:08.196261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcff1a1-b8ab-471a-8b59-f40eb77e4c24 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:08.201275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcff1a1-b8ab-471a-8b59-f40eb77e4c24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:08.211583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcff1a1-b8ab-471a-8b59-f40eb77e4c24 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:08.253693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 937/1876] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:08 2022] POST /identity/v3/users => generated 287 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:08.257826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3f760-9beb-4028-ac5c-784a77e141b6 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:08.290101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3f760-9beb-4028-ac5c-784a77e141b6 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:08.300894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3309a5-d559-499b-b65e-8180c92ee3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:08.302385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3309a5-d559-499b-b65e-8180c92ee3ad admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:08.302993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3309a5-d559-499b-b65e-8180c92ee3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:08.303591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3309a5-d559-499b-b65e-8180c92ee3ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:08.304728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3309a5-d559-499b-b65e-8180c92ee3ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:08.307242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3309a5-d559-499b-b65e-8180c92ee3ad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:08.327697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 940/1877] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:08.335538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcff1a1-b8ab-471a-8b59-f40eb77e4c24 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:08.357457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcff1a1-b8ab-471a-8b59-f40eb77e4c24 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:08.367968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c6393-475a-4cde-974d-087301b483f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:08.368771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c6393-475a-4cde-974d-087301b483f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:08.369424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c6393-475a-4cde-974d-087301b483f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:08.370254 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c6393-475a-4cde-974d-087301b483f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:08.371832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c6393-475a-4cde-974d-087301b483f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:08.375378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c6393-475a-4cde-974d-087301b483f9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:08.403161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 938/1878] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:08.407328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3309a5-d559-499b-b65e-8180c92ee3ad admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:08.422756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3309a5-d559-499b-b65e-8180c92ee3ad admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:08.430212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:08.431187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:08.431500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:08.431773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] PATH_INFO: `/v3/projects/17949674768145129d8b3589ed03a0f6/users/4c7a1546858743588652610172e2eb59/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:08.472053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17949674768145129d8b3589ed03a0f6, user_id=4c7a1546858743588652610172e2eb59, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:08.476420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:08.496762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 941/1879] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:08 2022] PUT /identity/v3/projects/17949674768145129d8b3589ed03a0f6/users/4c7a1546858743588652610172e2eb59/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:08.509971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb915f3e-f5fa-4159-a13c-eddfa33b3194 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:08.510698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb915f3e-f5fa-4159-a13c-eddfa33b3194 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:08.511699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb915f3e-f5fa-4159-a13c-eddfa33b3194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:08.584015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bb915f3e-f5fa-4159-a13c-eddfa33b3194 None None] MFA Rules not processed for user `4c7a1546858743588652610172e2eb59`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:08.585625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bb915f3e-f5fa-4159-a13c-eddfa33b3194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:08.610347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:08.728202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 939/1880] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:08.741991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:08.743756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:08.822300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9fcd7-d64f-411f-a03b-4620f8b81a85 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:08.826587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9dbba-0709-49e8-bef3-b2958b352899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:08.828159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9dbba-0709-49e8-bef3-b2958b352899 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:08.828900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9dbba-0709-49e8-bef3-b2958b352899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:08.829025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9dbba-0709-49e8-bef3-b2958b352899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:08.842462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9dbba-0709-49e8-bef3-b2958b352899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:08.844062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9dbba-0709-49e8-bef3-b2958b352899 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:08.857736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 942/1881] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:08.881173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb915f3e-f5fa-4159-a13c-eddfa33b3194 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:08.988302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb915f3e-f5fa-4159-a13c-eddfa33b3194 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:08.996829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:08.998607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:08.999136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:08.999646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] PATH_INFO: `/v3/users/4c7a1546858743588652610172e2eb59` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:09.001206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7a1546858743588652610172e2eb59)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:09.004548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:09.095304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7a1546858743588652610172e2eb59'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:09.109134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7a1546858743588652610172e2eb59'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:09.154857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7a1546858743588652610172e2eb59'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:09.158157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7a1546858743588652610172e2eb59'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:09.160260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7a1546858743588652610172e2eb59'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:09.179904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 940/1882] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:08 2022] DELETE /identity/v3/users/4c7a1546858743588652610172e2eb59 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:09.185179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf694f-3bdd-4680-853d-83e9d97727c7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:09.186838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf694f-3bdd-4680-853d-83e9d97727c7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:09.187385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf694f-3bdd-4680-853d-83e9d97727c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:09.210264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3dbf694f-3bdd-4680-853d-83e9d97727c7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:09.211783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3dbf694f-3bdd-4680-853d-83e9d97727c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:09.333046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 943/1883] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:09.346441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:09.348211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:09.394760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:09.415074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848d723-ca31-4e25-9243-2831da5c284c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:09.421748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad95da-e773-4d6a-abe4-a8b04533f677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:09.421748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad95da-e773-4d6a-abe4-a8b04533f677 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:09.422175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad95da-e773-4d6a-abe4-a8b04533f677 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:09.422175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad95da-e773-4d6a-abe4-a8b04533f677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:09.433401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad95da-e773-4d6a-abe4-a8b04533f677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:09.435097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad95da-e773-4d6a-abe4-a8b04533f677 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:09.448566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 941/1884] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:10.024787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf694f-3bdd-4680-853d-83e9d97727c7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:10.026793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3dbf694f-3bdd-4680-853d-83e9d97727c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:10.043466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf694f-3bdd-4680-853d-83e9d97727c7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:10.049544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:10.050910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.051420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.052046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] PATH_INFO: `/v3/projects/17949674768145129d8b3589ed03a0f6` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.054002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=17949674768145129d8b3589ed03a0f6)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:10.057773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:10.122448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:10.139633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17949674768145129d8b3589ed03a0f6', 'request_id': 'req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:10.151647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17949674768145129d8b3589ed03a0f6', 'request_id': 'req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:10.165636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17949674768145129d8b3589ed03a0f6', 'request_id': 'req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:10.184435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] The token cache is being invalidate because project 17949674768145129d8b3589ed03a0f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:10.189923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:10.192217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cdb01d9e-b020-4152-9f5c-79ed3d17ae2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:10.197997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 944/1885] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:10 2022] DELETE /identity/v3/projects/17949674768145129d8b3589ed03a0f6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:10.199616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37bd4-d721-4413-949e-8fa620762278 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.200235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37bd4-d721-4413-949e-8fa620762278 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.200235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b37bd4-d721-4413-949e-8fa620762278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.219984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f3b37bd4-d721-4413-949e-8fa620762278 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:10.221493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f3b37bd4-d721-4413-949e-8fa620762278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:10.302069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 942/1886] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:10.311961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e528-e29e-470c-b5d4-0389ff4f41b7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.312486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e528-e29e-470c-b5d4-0389ff4f41b7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.312898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e528-e29e-470c-b5d4-0389ff4f41b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.335887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-cc79e528-e29e-470c-b5d4-0389ff4f41b7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:10.337397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cc79e528-e29e-470c-b5d4-0389ff4f41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:10.347890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 945/1887] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:10.352843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b37bd4-d721-4413-949e-8fa620762278 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:10.375192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b37bd4-d721-4413-949e-8fa620762278 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:10.381291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:10.382047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.382575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.382868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.383641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:10.383970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:10.386972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:10.404745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 943/1888] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:10.411710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e528-e29e-470c-b5d4-0389ff4f41b7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:10.419852 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e528-e29e-470c-b5d4-0389ff4f41b7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:10.429728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a752cf-d4b9-4903-bc12-b98b2a0f3866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:10.430600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a752cf-d4b9-4903-bc12-b98b2a0f3866 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.430883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a752cf-d4b9-4903-bc12-b98b2a0f3866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.431170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a752cf-d4b9-4903-bc12-b98b2a0f3866 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.431902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a752cf-d4b9-4903-bc12-b98b2a0f3866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:10.433999 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a752cf-d4b9-4903-bc12-b98b2a0f3866 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:10.471732 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 946/1889] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:10 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:10.476183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:10.488347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188d017-95ca-41f8-880c-55c6170ea7a3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:10.561175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee8a48-a1c9-47b5-be57-12580274e5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:10.561919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8a48-a1c9-47b5-be57-12580274e5fa admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.562201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8a48-a1c9-47b5-be57-12580274e5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.562471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee8a48-a1c9-47b5-be57-12580274e5fa admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.563213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee8a48-a1c9-47b5-be57-12580274e5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:10.565426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee8a48-a1c9-47b5-be57-12580274e5fa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:10.591763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 944/1890] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:10 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:10.597140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a752cf-d4b9-4903-bc12-b98b2a0f3866 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:10.612595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a752cf-d4b9-4903-bc12-b98b2a0f3866 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:10.620936 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2f81-f701-4c77-8e17-3cbc3fc9a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:10.622004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2f81-f701-4c77-8e17-3cbc3fc9a2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.622479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2f81-f701-4c77-8e17-3cbc3fc9a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.622963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2f81-f701-4c77-8e17-3cbc3fc9a2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.624005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb2f81-f701-4c77-8e17-3cbc3fc9a2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:10.626883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb2f81-f701-4c77-8e17-3cbc3fc9a2c7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:10.652453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 947/1891] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:10.657594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee8a48-a1c9-47b5-be57-12580274e5fa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:10.670002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee8a48-a1c9-47b5-be57-12580274e5fa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:10.675823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7ea08-a209-4ec4-8f86-0fa129b3e21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:10.676543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7ea08-a209-4ec4-8f86-0fa129b3e21d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.676827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7ea08-a209-4ec4-8f86-0fa129b3e21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.677105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7ea08-a209-4ec4-8f86-0fa129b3e21d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.677897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7ea08-a209-4ec4-8f86-0fa129b3e21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:10.679600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7ea08-a209-4ec4-8f86-0fa129b3e21d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:10.691756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 945/1892] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:10.696824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2f81-f701-4c77-8e17-3cbc3fc9a2c7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:10.710366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2f81-f701-4c77-8e17-3cbc3fc9a2c7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:10.716586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:10.717556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.717922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.718215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] PATH_INFO: `/v3/projects/2bba9fd8306e43b6b1cbc2dd4a789736/users/2865db5146784ee188001703d85d8809/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.738628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bba9fd8306e43b6b1cbc2dd4a789736, user_id=2865db5146784ee188001703d85d8809, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:10.742077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:10.764757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 948/1893] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:10 2022] PUT /identity/v3/projects/2bba9fd8306e43b6b1cbc2dd4a789736/users/2865db5146784ee188001703d85d8809/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:10.773109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5679-a340-43b0-b45a-32569a74f46c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:10.773625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5679-a340-43b0-b45a-32569a74f46c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:10.774169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5679-a340-43b0-b45a-32569a74f46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:10.808041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5a7e5679-a340-43b0-b45a-32569a74f46c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:10.809340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5a7e5679-a340-43b0-b45a-32569a74f46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:10.871271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:10.908142 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 946/1894] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:10.925140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:10.928000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:11.009340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42ed3d-9f02-465a-9829-61526890e7b9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:11.013553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:11.014582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:11.015235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:11.015825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:11.028931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:11.030878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:11.043876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 949/1895] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:12.086488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4b2e-024b-4ed8-82e8-954fcba760d6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:12.087388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4b2e-024b-4ed8-82e8-954fcba760d6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:12.087687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4b2e-024b-4ed8-82e8-954fcba760d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:12.119362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7f6b4b2e-024b-4ed8-82e8-954fcba760d6 None None] MFA Rules not processed for user `2865db5146784ee188001703d85d8809`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:12.120779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7f6b4b2e-024b-4ed8-82e8-954fcba760d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:12.205341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 947/1896] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:12.217006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:12.218590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:12.302051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ec7df-d594-4ff9-9873-1b056d85305b service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:12.306787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:12.307717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:12.308247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:12.308397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:12.318430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:12.319951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:12.330784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 950/1897] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:13.769246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655552c1-68ae-476a-ab38-668f60ad136d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:13.769938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655552c1-68ae-476a-ab38-668f60ad136d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:13.769938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655552c1-68ae-476a-ab38-668f60ad136d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:13.791639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-655552c1-68ae-476a-ab38-668f60ad136d None None] MFA Rules not processed for user `2865db5146784ee188001703d85d8809`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:13.793250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-655552c1-68ae-476a-ab38-668f60ad136d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:13.807405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 948/1898] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:16.928407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:16.930793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:16.948392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721604f5-7316-4b36-a9a6-b2eb447a6bf1 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:16.958554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:16.959747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:16.960221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:16.960674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] PATH_INFO: `/v3/users/bea9d5e106d742b19bbd7b3d80557e58` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:16.964634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bea9d5e106d742b19bbd7b3d80557e58)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:16.967154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:16.969302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:17.112543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bea9d5e106d742b19bbd7b3d80557e58'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:17.138375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bea9d5e106d742b19bbd7b3d80557e58'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:17.188121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bea9d5e106d742b19bbd7b3d80557e58'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:17.189153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bea9d5e106d742b19bbd7b3d80557e58'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:17.190397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bea9d5e106d742b19bbd7b3d80557e58'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:17.228004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 951/1899] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:16 2022] DELETE /identity/v3/users/bea9d5e106d742b19bbd7b3d80557e58 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:17.235201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01e2f4-51b1-4c80-bc18-709c985425b9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:17.235792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01e2f4-51b1-4c80-bc18-709c985425b9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:17.235932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01e2f4-51b1-4c80-bc18-709c985425b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:17.275956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2e01e2f4-51b1-4c80-bc18-709c985425b9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:17.280423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2e01e2f4-51b1-4c80-bc18-709c985425b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:17.389632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 949/1900] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:17.641605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:17.644530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:17.802257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05871cc-6ba3-48ac-afb4-cef94b5f6edd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:17.812696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3e9d5-7025-40cd-99fe-e872bf2b4195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:17.814201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3e9d5-7025-40cd-99fe-e872bf2b4195 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:17.814870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3e9d5-7025-40cd-99fe-e872bf2b4195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:17.815453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3e9d5-7025-40cd-99fe-e872bf2b4195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:17.839347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3e9d5-7025-40cd-99fe-e872bf2b4195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:17.841237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3e9d5-7025-40cd-99fe-e872bf2b4195 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:17.856109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 952/1901] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:19.105280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01e2f4-51b1-4c80-bc18-709c985425b9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:19.109171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2e01e2f4-51b1-4c80-bc18-709c985425b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:19.145259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01e2f4-51b1-4c80-bc18-709c985425b9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:19.153447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:19.154761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:19.155233 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:19.155769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] PATH_INFO: `/v3/projects/1f807109c12a48e492c65299a9e8bf21` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:19.158023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f807109c12a48e492c65299a9e8bf21)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:19.161942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:19.255322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f807109c12a48e492c65299a9e8bf21', 'request_id': 'req-88b36a09-2c7a-4597-beed-02bfe847886e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.277439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f807109c12a48e492c65299a9e8bf21', 'request_id': 'req-88b36a09-2c7a-4597-beed-02bfe847886e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.286677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f807109c12a48e492c65299a9e8bf21', 'request_id': 'req-88b36a09-2c7a-4597-beed-02bfe847886e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.294995 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:19.324528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] The token cache is being invalidate because project 1f807109c12a48e492c65299a9e8bf21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:19.327070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.327818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-88b36a09-2c7a-4597-beed-02bfe847886e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.335963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 950/1902] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:19 2022] DELETE /identity/v3/projects/1f807109c12a48e492c65299a9e8bf21 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:19.339905 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3e9d5-7025-40cd-99fe-e872bf2b4195 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:19.351076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3e9d5-7025-40cd-99fe-e872bf2b4195 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:19.436702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:19.437834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:19.438135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:19.438432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] PATH_INFO: `/v3/users/0eeeeea271134cd9b8ccdc75fcdf71f8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:19.440983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eeeeea271134cd9b8ccdc75fcdf71f8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:19.446770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:19.492054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:19.539235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eeeeea271134cd9b8ccdc75fcdf71f8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.561508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eeeeea271134cd9b8ccdc75fcdf71f8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.603230 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eeeeea271134cd9b8ccdc75fcdf71f8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.605052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eeeeea271134cd9b8ccdc75fcdf71f8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.609227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-58106298-f78e-49f9-9197-aa7d6d1790d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eeeeea271134cd9b8ccdc75fcdf71f8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:19.639015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 953/1903] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:19 2022] DELETE /identity/v3/users/0eeeeea271134cd9b8ccdc75fcdf71f8 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:20.155608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f6ddd-2f14-4c3f-bfe9-9d0c099c5faf None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:20.156329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f6ddd-2f14-4c3f-bfe9-9d0c099c5faf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:20.156999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f6ddd-2f14-4c3f-bfe9-9d0c099c5faf None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:20.159086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 951/1904] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:20:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:20:20.178958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c174-86d0-4932-a744-7d420d32fda3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:20.179344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c174-86d0-4932-a744-7d420d32fda3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:20.180369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1c174-86d0-4932-a744-7d420d32fda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:20.200647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7bf1c174-86d0-4932-a744-7d420d32fda3 None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:20.201921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7bf1c174-86d0-4932-a744-7d420d32fda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:20.283650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 954/1905] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:20:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:20.307178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f6ddd-2f14-4c3f-bfe9-9d0c099c5faf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:20.310904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-055f6ddd-2f14-4c3f-bfe9-9d0c099c5faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:20.338249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f6ddd-2f14-4c3f-bfe9-9d0c099c5faf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:20.348160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d51e1-44e2-48d5-bf8a-cd960ea762a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:20.348944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d51e1-44e2-48d5-bf8a-cd960ea762a7 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:20.349294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d51e1-44e2-48d5-bf8a-cd960ea762a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:20.349637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d51e1-44e2-48d5-bf8a-cd960ea762a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:20.361467 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d51e1-44e2-48d5-bf8a-cd960ea762a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:20.363701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d51e1-44e2-48d5-bf8a-cd960ea762a7 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:20.375243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 952/1906] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:21.207476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c174-86d0-4932-a744-7d420d32fda3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:21.209260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7bf1c174-86d0-4932-a744-7d420d32fda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:21.291419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1c174-86d0-4932-a744-7d420d32fda3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:21.296644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:21.297697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.298047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.298905 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] PATH_INFO: `/v3/projects/4d73bf99f59742c383528c06073daa10` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.301165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d73bf99f59742c383528c06073daa10)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:21.303202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:21.361046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:21.377702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d73bf99f59742c383528c06073daa10', 'request_id': 'req-75f775fb-f748-4619-83ad-cfdf1715ec82'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.390667 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d73bf99f59742c383528c06073daa10', 'request_id': 'req-75f775fb-f748-4619-83ad-cfdf1715ec82'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.415118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d73bf99f59742c383528c06073daa10', 'request_id': 'req-75f775fb-f748-4619-83ad-cfdf1715ec82'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.435772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] The token cache is being invalidate because project 4d73bf99f59742c383528c06073daa10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:21.436131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.436909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.439395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 955/1907] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:21 2022] DELETE /identity/v3/projects/4d73bf99f59742c383528c06073daa10 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:21.444617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad3469-043f-41ff-947c-6375429f3b48 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.446032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad3469-043f-41ff-947c-6375429f3b48 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.446364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad3469-043f-41ff-947c-6375429f3b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.467087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-56ad3469-043f-41ff-947c-6375429f3b48 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:21.468319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-56ad3469-043f-41ff-947c-6375429f3b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:21.526251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:21.528047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:21.570793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 953/1908] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:21.582499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda9e0a-cd74-497a-9fb3-a603e3b3eb6a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.582799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda9e0a-cd74-497a-9fb3-a603e3b3eb6a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.583201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda9e0a-cd74-497a-9fb3-a603e3b3eb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.605727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cbda9e0a-cd74-497a-9fb3-a603e3b3eb6a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:21.607101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cbda9e0a-cd74-497a-9fb3-a603e3b3eb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:21.617083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 954/1909] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:21.621515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda9e0a-cd74-497a-9fb3-a603e3b3eb6a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:21.623457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f775fb-f748-4619-83ad-cfdf1715ec82 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:21.629763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:21.629763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.630064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.630860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] PATH_INFO: `/v3/users/c26e061da2844459990e1f0a79869468` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.639201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26e061da2844459990e1f0a79869468)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:21.640579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda9e0a-cd74-497a-9fb3-a603e3b3eb6a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:21.643050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:21.647769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:21.649169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.649169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.649169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.649779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:21.650092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:21.651714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:21.665272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 955/1910] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:20:21.670139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:21.681441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10905c8-19e5-49e6-961e-caa95dbf7dd2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:21.687229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8192-fcfc-4ffb-bdfb-254597d3d66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:21.688586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8192-fcfc-4ffb-bdfb-254597d3d66a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.688877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8192-fcfc-4ffb-bdfb-254597d3d66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.689159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa8192-fcfc-4ffb-bdfb-254597d3d66a admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.690851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa8192-fcfc-4ffb-bdfb-254597d3d66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:21.692592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa8192-fcfc-4ffb-bdfb-254597d3d66a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:21.717413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26e061da2844459990e1f0a79869468'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.726473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 956/1911] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:21 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:21.728733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26e061da2844459990e1f0a79869468'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.730835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8192-fcfc-4ffb-bdfb-254597d3d66a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:21.755238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26e061da2844459990e1f0a79869468'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.756347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26e061da2844459990e1f0a79869468'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.757264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26e061da2844459990e1f0a79869468'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:21.776207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 956/1912] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:21 2022] DELETE /identity/v3/users/c26e061da2844459990e1f0a79869468 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:21.820458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa8192-fcfc-4ffb-bdfb-254597d3d66a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:21.873631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:21.893211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fbdf3-479d-487d-86b6-d927d1bddd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:21.893954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fbdf3-479d-487d-86b6-d927d1bddd2e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.894238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fbdf3-479d-487d-86b6-d927d1bddd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.894519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fbdf3-479d-487d-86b6-d927d1bddd2e admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.895233 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fbdf3-479d-487d-86b6-d927d1bddd2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:21.897323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fbdf3-479d-487d-86b6-d927d1bddd2e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:21.921917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 957/1913] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:21 2022] POST /identity/v3/users => generated 301 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:21.926329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:21.927919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:21.944095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63676a-4f5e-420e-99ff-3913f0930153 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:21.949838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab998a5c-b448-4d10-87c5-95fe587e62a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:21.950253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab998a5c-b448-4d10-87c5-95fe587e62a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.950793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab998a5c-b448-4d10-87c5-95fe587e62a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.951033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab998a5c-b448-4d10-87c5-95fe587e62a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.952169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab998a5c-b448-4d10-87c5-95fe587e62a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:21.953874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab998a5c-b448-4d10-87c5-95fe587e62a7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:21.966654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 957/1914] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:21.970347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fbdf3-479d-487d-86b6-d927d1bddd2e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:21.981441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fbdf3-479d-487d-86b6-d927d1bddd2e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:21.986490 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:21.987191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:21.987451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:21.987855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:21.988608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:21.990601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.001584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 958/1915] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:22.006427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab998a5c-b448-4d10-87c5-95fe587e62a7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.019233 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab998a5c-b448-4d10-87c5-95fe587e62a7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.025537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1867e-8cd0-4e50-b672-5b350011b48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.026490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1867e-8cd0-4e50-b672-5b350011b48d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.026820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1867e-8cd0-4e50-b672-5b350011b48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.027697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1867e-8cd0-4e50-b672-5b350011b48d admin admin] PATH_INFO: `/v3/projects/b7ce7b4c18964df0b1e8da760bf15e7a/users/e8ecf961de64466280fd2fc325bb1f59/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.052558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1867e-8cd0-4e50-b672-5b350011b48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ce7b4c18964df0b1e8da760bf15e7a, user_id=e8ecf961de64466280fd2fc325bb1f59, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.055591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1867e-8cd0-4e50-b672-5b350011b48d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.059719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.062049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.077340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 958/1916] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:22 2022] PUT /identity/v3/projects/b7ce7b4c18964df0b1e8da760bf15e7a/users/e8ecf961de64466280fd2fc325bb1f59/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:22.083718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1306fb7-c65b-4900-90d8-db7d8b6edec7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.089317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f731ab-f766-4ef2-ae07-cfdd3870aad3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.089968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f731ab-f766-4ef2-ae07-cfdd3870aad3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.090429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f731ab-f766-4ef2-ae07-cfdd3870aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.111814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:22.122840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-20f731ab-f766-4ef2-ae07-cfdd3870aad3 None None] MFA Rules not processed for user `e8ecf961de64466280fd2fc325bb1f59`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:22.124813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-20f731ab-f766-4ef2-ae07-cfdd3870aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.158426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.159656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.159949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.160229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] PATH_INFO: `/v3/projects/f12604fad21641819b829c553b63398b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.161541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f12604fad21641819b829c553b63398b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.163582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.198626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 959/1917] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:22.203193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f731ab-f766-4ef2-ae07-cfdd3870aad3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.211356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f12604fad21641819b829c553b63398b', 'request_id': 'req-b292856f-5159-4b9d-b475-6365379cc04d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.214868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f731ab-f766-4ef2-ae07-cfdd3870aad3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.221131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f12604fad21641819b829c553b63398b', 'request_id': 'req-b292856f-5159-4b9d-b475-6365379cc04d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.229406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f12604fad21641819b829c553b63398b', 'request_id': 'req-b292856f-5159-4b9d-b475-6365379cc04d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.243507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] The token cache is being invalidate because project f12604fad21641819b829c553b63398b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:22.243969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.244686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.251963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 959/1918] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:22 2022] DELETE /identity/v3/projects/f12604fad21641819b829c553b63398b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:22.253496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.256664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.291397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.292800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.293093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.293394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.294135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.296179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.331738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 960/1919] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:22 2022] POST /identity/v3/projects => generated 399 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:22.336486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.338401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.342135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:22.356402 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292856f-5159-4b9d-b475-6365379cc04d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.421127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adab5f6-8ed6-481c-97cd-7469a16a6f1b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.426241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.427189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c0077-81ea-4756-bcc2-2ad42438542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.427414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.427769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.427986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c0077-81ea-4756-bcc2-2ad42438542f admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.428176 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] PATH_INFO: `/v3/users/753a861b73bf4a06b45479bcbd0ae496` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.428298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c0077-81ea-4756-bcc2-2ad42438542f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.428537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c0077-81ea-4756-bcc2-2ad42438542f admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.429577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c0077-81ea-4756-bcc2-2ad42438542f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.430175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] RBAC: Authorizing `identity:delete_user(user_id=753a861b73bf4a06b45479bcbd0ae496)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.431893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017c0077-81ea-4756-bcc2-2ad42438542f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.432264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.474732 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 961/1920] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:22 2022] POST /identity/v3/users => generated 306 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:22.478833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c0077-81ea-4756-bcc2-2ad42438542f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.490846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017c0077-81ea-4756-bcc2-2ad42438542f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.497104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dc3ff-8892-4708-8bdb-ee3d8e6a97f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.497460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753a861b73bf4a06b45479bcbd0ae496'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.497967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc3ff-8892-4708-8bdb-ee3d8e6a97f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.498070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc3ff-8892-4708-8bdb-ee3d8e6a97f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.498460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dc3ff-8892-4708-8bdb-ee3d8e6a97f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.499189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dc3ff-8892-4708-8bdb-ee3d8e6a97f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.500809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dc3ff-8892-4708-8bdb-ee3d8e6a97f0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.506440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753a861b73bf4a06b45479bcbd0ae496'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.515727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 962/1921] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:22.516763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dc3ff-8892-4708-8bdb-ee3d8e6a97f0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.519757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753a861b73bf4a06b45479bcbd0ae496'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.520494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753a861b73bf4a06b45479bcbd0ae496'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.521123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753a861b73bf4a06b45479bcbd0ae496'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.523634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:22.528711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dc3ff-8892-4708-8bdb-ee3d8e6a97f0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.534862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 960/1922] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:22 2022] DELETE /identity/v3/users/753a861b73bf4a06b45479bcbd0ae496 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:22.535984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.536896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.537177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.537460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] PATH_INFO: `/v3/projects/79a292322b8d471988ba77c75c7c4d53/users/cadedb16989543d4b9c929d7cb435585/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.554805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a292322b8d471988ba77c75c7c4d53, user_id=cadedb16989543d4b9c929d7cb435585, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.557532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.572071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 963/1923] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:22 2022] PUT /identity/v3/projects/79a292322b8d471988ba77c75c7c4d53/users/cadedb16989543d4b9c929d7cb435585/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:22.577408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.581247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.667592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1b6f4-532e-4bc2-9dfa-c46d94faacac admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.694284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:22.735731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.739374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.756581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf616b29-5e71-4a78-80e6-003396530d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.757290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf616b29-5e71-4a78-80e6-003396530d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.757579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf616b29-5e71-4a78-80e6-003396530d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.757898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf616b29-5e71-4a78-80e6-003396530d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.758641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf616b29-5e71-4a78-80e6-003396530d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.760315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf616b29-5e71-4a78-80e6-003396530d46 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.773701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 961/1924] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:22.776579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf616b29-5e71-4a78-80e6-003396530d46 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.787278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf616b29-5e71-4a78-80e6-003396530d46 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.792346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b636c9-4c7b-41f2-a51c-57541ac04b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.793350 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b636c9-4c7b-41f2-a51c-57541ac04b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.793730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b636c9-4c7b-41f2-a51c-57541ac04b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.794101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b636c9-4c7b-41f2-a51c-57541ac04b1d admin admin] PATH_INFO: `/v3/projects/79a292322b8d471988ba77c75c7c4d53/users/cadedb16989543d4b9c929d7cb435585/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.796276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b636c9-4c7b-41f2-a51c-57541ac04b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a292322b8d471988ba77c75c7c4d53, user_id=cadedb16989543d4b9c929d7cb435585, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.799103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b636c9-4c7b-41f2-a51c-57541ac04b1d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.810280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 962/1925] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:22 2022] PUT /identity/v3/projects/79a292322b8d471988ba77c75c7c4d53/users/cadedb16989543d4b9c929d7cb435585/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:22.820671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426f677-b734-4925-9c68-62c2cbcae648 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.820971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426f677-b734-4925-9c68-62c2cbcae648 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.821236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426f677-b734-4925-9c68-62c2cbcae648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.831857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa1121-f1c4-450e-89d6-9e974cae957c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:22.848361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c426f677-b734-4925-9c68-62c2cbcae648 None None] MFA Rules not processed for user `cadedb16989543d4b9c929d7cb435585`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:22.849615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c426f677-b734-4925-9c68-62c2cbcae648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.899717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:22.900596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.900881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.901154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] PATH_INFO: `/v3/projects/70fd0f5ab218464eb1a9517b8b4b28b2` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.902395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70fd0f5ab218464eb1a9517b8b4b28b2)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:22.904325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:22.921903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 963/1926] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:22 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:22.933457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426f677-b734-4925-9c68-62c2cbcae648 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:22.935471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c426f677-b734-4925-9c68-62c2cbcae648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:22.952586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70fd0f5ab218464eb1a9517b8b4b28b2', 'request_id': 'req-2372431a-a692-4742-ae42-3a87799d57a0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.959212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70fd0f5ab218464eb1a9517b8b4b28b2', 'request_id': 'req-2372431a-a692-4742-ae42-3a87799d57a0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.967441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70fd0f5ab218464eb1a9517b8b4b28b2', 'request_id': 'req-2372431a-a692-4742-ae42-3a87799d57a0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.979793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] The token cache is being invalidate because project 70fd0f5ab218464eb1a9517b8b4b28b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:22.980001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.980657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2372431a-a692-4742-ae42-3a87799d57a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:22.982290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 964/1927] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:22 2022] DELETE /identity/v3/projects/70fd0f5ab218464eb1a9517b8b4b28b2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:22.987152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a844e-2c16-44b0-9660-eed02b18734a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:22.987436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a844e-2c16-44b0-9660-eed02b18734a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:22.987714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a844e-2c16-44b0-9660-eed02b18734a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:22.996140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426f677-b734-4925-9c68-62c2cbcae648 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.006807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6c2a844e-2c16-44b0-9660-eed02b18734a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:23.008044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6c2a844e-2c16-44b0-9660-eed02b18734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:23.020701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:23.048686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.049380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.049782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.050062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.051986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:23.057128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:23.072811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 965/1928] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:23.080813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd383f56-b4ac-4ca3-b576-9f1d357d9ae8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.081082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd383f56-b4ac-4ca3-b576-9f1d357d9ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.081356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd383f56-b4ac-4ca3-b576-9f1d357d9ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.100289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-cd383f56-b4ac-4ca3-b576-9f1d357d9ae8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:23.101524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cd383f56-b4ac-4ca3-b576-9f1d357d9ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:23.109725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 966/1929] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:23.114235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd383f56-b4ac-4ca3-b576-9f1d357d9ae8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.119722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.121170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.123331 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd383f56-b4ac-4ca3-b576-9f1d357d9ae8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.128212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.128880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.129166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.129443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.130280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:23.130594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.131210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 964/1930] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:22 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:23.132242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.144770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 967/1931] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:20:23.149583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.161400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a409c-20e9-406c-8416-7983e87800e9 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.167021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.167713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.168153 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.168436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.169268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.171270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.201483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 965/1932] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:23 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:23.207008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.217864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.219602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:23.224382 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87ae56-3ba8-48de-9804-4183fce0e22d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.286887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cccb1-3b75-46f2-a4f8-7618fad79dd2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.290746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.291514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.291865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.292152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.293011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ec8b4-6afb-4701-849c-29930e4839d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.293744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec8b4-6afb-4701-849c-29930e4839d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.294028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec8b4-6afb-4701-849c-29930e4839d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.294302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ec8b4-6afb-4701-849c-29930e4839d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.295043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ec8b4-6afb-4701-849c-29930e4839d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.295345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:23.297100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ec8b4-6afb-4701-849c-29930e4839d7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.326784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 968/1933] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:23 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:23.328214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ec8b4-6afb-4701-849c-29930e4839d7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.338922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ec8b4-6afb-4701-849c-29930e4839d7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.346283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04918-9cfc-4cde-8535-d4e6a416106f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.346973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04918-9cfc-4cde-8535-d4e6a416106f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.347278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04918-9cfc-4cde-8535-d4e6a416106f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.347709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f04918-9cfc-4cde-8535-d4e6a416106f admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.348447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f04918-9cfc-4cde-8535-d4e6a416106f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.350135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f04918-9cfc-4cde-8535-d4e6a416106f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.367025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 969/1934] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:23.370599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.371199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04918-9cfc-4cde-8535-d4e6a416106f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.372095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.382376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 966/1935] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:20:23.382708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f04918-9cfc-4cde-8535-d4e6a416106f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.387676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e57d0-4388-431b-8506-326e543e5130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.388323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e57d0-4388-431b-8506-326e543e5130 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.388618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e57d0-4388-431b-8506-326e543e5130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.388870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e57d0-4388-431b-8506-326e543e5130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.389603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e57d0-4388-431b-8506-326e543e5130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.391242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e57d0-4388-431b-8506-326e543e5130 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.404153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 970/1936] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:23.409291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.420817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baf0b5-b926-4012-af12-01a2066423c1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.428548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.429479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.429828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.430123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] PATH_INFO: `/v3/projects/504d3443eaf7493d8b12e3a6c8d4618a/users/65a601b3efee40369364c5e42221f060/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.447975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=504d3443eaf7493d8b12e3a6c8d4618a, user_id=65a601b3efee40369364c5e42221f060, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.451165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.468044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 967/1937] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:23 2022] PUT /identity/v3/projects/504d3443eaf7493d8b12e3a6c8d4618a/users/65a601b3efee40369364c5e42221f060/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:23.485143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b952f-d485-49a6-a26f-11ec43aefd70 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.485143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b952f-d485-49a6-a26f-11ec43aefd70 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.485143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b952f-d485-49a6-a26f-11ec43aefd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.533724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-267b952f-d485-49a6-a26f-11ec43aefd70 None None] MFA Rules not processed for user `65a601b3efee40369364c5e42221f060`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:23.535014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-267b952f-d485-49a6-a26f-11ec43aefd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:23.618879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:23.647049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 971/1938] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:23.654184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.657243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:23.670447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:23.750409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b133d8-747f-49a0-9375-23ffcd2414cf admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.764532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944e9be-49c5-4ac8-893e-8d0257742552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.765228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e9be-49c5-4ac8-893e-8d0257742552 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.765514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e9be-49c5-4ac8-893e-8d0257742552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.767532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944e9be-49c5-4ac8-893e-8d0257742552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.797979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944e9be-49c5-4ac8-893e-8d0257742552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.799425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944e9be-49c5-4ac8-893e-8d0257742552 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.826184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 968/1939] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:23.848279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b952f-d485-49a6-a26f-11ec43aefd70 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:23.914116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b952f-d485-49a6-a26f-11ec43aefd70 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:23.918908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:23.919867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:23.920215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:23.920559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] PATH_INFO: `/v3/users/65a601b3efee40369364c5e42221f060` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:23.921869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a601b3efee40369364c5e42221f060)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:23.923871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:23.972622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a601b3efee40369364c5e42221f060'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:23.981826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a601b3efee40369364c5e42221f060'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:23.997865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a601b3efee40369364c5e42221f060'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:23.998806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a601b3efee40369364c5e42221f060'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:23.999438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a601b3efee40369364c5e42221f060'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:24.013545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 972/1940] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:23 2022] DELETE /identity/v3/users/65a601b3efee40369364c5e42221f060 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:24.018507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e42cb-f248-4125-b180-a347a8afcf7d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:24.018795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e42cb-f248-4125-b180-a347a8afcf7d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:24.019077 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e42cb-f248-4125-b180-a347a8afcf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:24.037521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f60e42cb-f248-4125-b180-a347a8afcf7d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:24.038777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f60e42cb-f248-4125-b180-a347a8afcf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:24.109426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 969/1941] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:24.121881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:24.123328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:24.183588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c50843-bdeb-443f-b059-f8839fccc3ed admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:24.187669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a197f30-578b-4acb-b759-3e42776c61db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:24.188316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a197f30-578b-4acb-b759-3e42776c61db service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:24.188597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a197f30-578b-4acb-b759-3e42776c61db service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:24.188871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a197f30-578b-4acb-b759-3e42776c61db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:24.199172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a197f30-578b-4acb-b759-3e42776c61db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:24.200646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a197f30-578b-4acb-b759-3e42776c61db service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:24.211185 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 973/1942] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:24.530993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e42cb-f248-4125-b180-a347a8afcf7d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:24.532798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f60e42cb-f248-4125-b180-a347a8afcf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:24.551211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e42cb-f248-4125-b180-a347a8afcf7d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:24.557748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:24.558741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:24.559102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:24.559447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] PATH_INFO: `/v3/projects/504d3443eaf7493d8b12e3a6c8d4618a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:24.564412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=504d3443eaf7493d8b12e3a6c8d4618a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:24.566582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:24.628752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '504d3443eaf7493d8b12e3a6c8d4618a', 'request_id': 'req-4be34969-c15d-49e6-b359-55a2703d84dd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:24.640078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '504d3443eaf7493d8b12e3a6c8d4618a', 'request_id': 'req-4be34969-c15d-49e6-b359-55a2703d84dd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:24.647677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '504d3443eaf7493d8b12e3a6c8d4618a', 'request_id': 'req-4be34969-c15d-49e6-b359-55a2703d84dd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:24.659169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] The token cache is being invalidate because project 504d3443eaf7493d8b12e3a6c8d4618a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:24.660090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:24.660243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-4be34969-c15d-49e6-b359-55a2703d84dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:24.662068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 970/1943] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:24 2022] DELETE /identity/v3/projects/504d3443eaf7493d8b12e3a6c8d4618a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:24.666914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad02d9-92b2-4577-8f8d-b6a5a4eff8ce None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:24.667497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad02d9-92b2-4577-8f8d-b6a5a4eff8ce None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:24.667497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad02d9-92b2-4577-8f8d-b6a5a4eff8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:24.686024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-73ad02d9-92b2-4577-8f8d-b6a5a4eff8ce None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:24.687293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-73ad02d9-92b2-4577-8f8d-b6a5a4eff8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:24.700378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:24.758167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 974/1944] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:24.766010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6116a3-141a-44c7-97d3-eaf30625ce35 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:24.766288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6116a3-141a-44c7-97d3-eaf30625ce35 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:24.766625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6116a3-141a-44c7-97d3-eaf30625ce35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:24.789502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3c6116a3-141a-44c7-97d3-eaf30625ce35 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:24.791154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3c6116a3-141a-44c7-97d3-eaf30625ce35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:24.808346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 971/1945] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:24.810814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad02d9-92b2-4577-8f8d-b6a5a4eff8ce None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:24.830828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad02d9-92b2-4577-8f8d-b6a5a4eff8ce None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:24.840932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:24.842910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:24.843892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:24.844599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:24.846161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:24.846989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:24.850939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:24.882404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 975/1946] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:24.886465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6116a3-141a-44c7-97d3-eaf30625ce35 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:24.898604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6116a3-141a-44c7-97d3-eaf30625ce35 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:24.904040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574516c5-259b-4ca5-9235-748335a9bb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:24.904941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574516c5-259b-4ca5-9235-748335a9bb8f admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:24.905391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574516c5-259b-4ca5-9235-748335a9bb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:24.905895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574516c5-259b-4ca5-9235-748335a9bb8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:24.906619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574516c5-259b-4ca5-9235-748335a9bb8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:24.909228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574516c5-259b-4ca5-9235-748335a9bb8f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:24.951026 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 972/1947] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:24 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:24.952197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:24.969835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d311c-8c17-496a-b751-4f6d883b24ba admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:25.062015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b3954-5a3e-4de3-a40e-6e7c0406d498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:25.062796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3954-5a3e-4de3-a40e-6e7c0406d498 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.063089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3954-5a3e-4de3-a40e-6e7c0406d498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.063379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3954-5a3e-4de3-a40e-6e7c0406d498 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.064166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b3954-5a3e-4de3-a40e-6e7c0406d498 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:25.066722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b3954-5a3e-4de3-a40e-6e7c0406d498 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:25.089473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 976/1948] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:24 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:25.094046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574516c5-259b-4ca5-9235-748335a9bb8f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:25.110496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574516c5-259b-4ca5-9235-748335a9bb8f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:25.116312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1d1d0-92ad-4c42-8894-ceb5ff07a684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:25.117007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d1d0-92ad-4c42-8894-ceb5ff07a684 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.117295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d1d0-92ad-4c42-8894-ceb5ff07a684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.117573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1d1d0-92ad-4c42-8894-ceb5ff07a684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.118520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1d1d0-92ad-4c42-8894-ceb5ff07a684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:25.120147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1d1d0-92ad-4c42-8894-ceb5ff07a684 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:25.121770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:25.137132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 973/1949] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:25.137982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b3954-5a3e-4de3-a40e-6e7c0406d498 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:25.167998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b3954-5a3e-4de3-a40e-6e7c0406d498 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:25.173184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420722bc-ff6c-4268-ae31-169c492b9a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:25.174257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420722bc-ff6c-4268-ae31-169c492b9a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.174610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420722bc-ff6c-4268-ae31-169c492b9a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.174945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420722bc-ff6c-4268-ae31-169c492b9a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.175745 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420722bc-ff6c-4268-ae31-169c492b9a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:25.177430 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420722bc-ff6c-4268-ae31-169c492b9a12 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:25.188438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 977/1950] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:25.193277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1d1d0-92ad-4c42-8894-ceb5ff07a684 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:25.203626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1d1d0-92ad-4c42-8894-ceb5ff07a684 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:25.208947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:25.210058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.210420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.210690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] PATH_INFO: `/v3/projects/cfe294a0659d4203971216d53f516dc2/users/f79c5ac46cf941bfa3691be50fb1e34b/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.226271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe294a0659d4203971216d53f516dc2, user_id=f79c5ac46cf941bfa3691be50fb1e34b, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:25.228936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:25.244388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 974/1951] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:25 2022] PUT /identity/v3/projects/cfe294a0659d4203971216d53f516dc2/users/f79c5ac46cf941bfa3691be50fb1e34b/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:25.255364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfca1b9-eb9f-42dc-a702-846d64367cfe None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.255651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfca1b9-eb9f-42dc-a702-846d64367cfe None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.255926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfca1b9-eb9f-42dc-a702-846d64367cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.285001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-dcfca1b9-eb9f-42dc-a702-846d64367cfe None None] MFA Rules not processed for user `f79c5ac46cf941bfa3691be50fb1e34b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:25.286287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dcfca1b9-eb9f-42dc-a702-846d64367cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:25.355377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 978/1952] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:25.367735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:25.369854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:25.434364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cfb8f-3c70-4816-9846-51e4a3887c7f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:25.440448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda1657-2a95-4d25-84c2-476bf36e431e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:25.441117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda1657-2a95-4d25-84c2-476bf36e431e service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:25.441405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda1657-2a95-4d25-84c2-476bf36e431e service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:25.441724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda1657-2a95-4d25-84c2-476bf36e431e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:25.451803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda1657-2a95-4d25-84c2-476bf36e431e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:25.453280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda1657-2a95-4d25-84c2-476bf36e431e service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:25.465068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 975/1953] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:27.798326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfca1b9-eb9f-42dc-a702-846d64367cfe None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:27.800210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dcfca1b9-eb9f-42dc-a702-846d64367cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:27.817135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:27.881376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfca1b9-eb9f-42dc-a702-846d64367cfe None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:27.885998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:27.886891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:27.887172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:27.887449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] PATH_INFO: `/v3/users/e8ecf961de64466280fd2fc325bb1f59` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:27.888663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ecf961de64466280fd2fc325bb1f59)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:27.890668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:27.947458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ecf961de64466280fd2fc325bb1f59'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:27.957920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ecf961de64466280fd2fc325bb1f59'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:27.976065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ecf961de64466280fd2fc325bb1f59'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:27.977008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ecf961de64466280fd2fc325bb1f59'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:27.977997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ecf961de64466280fd2fc325bb1f59'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:27.995072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 979/1954] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:27 2022] DELETE /identity/v3/users/e8ecf961de64466280fd2fc325bb1f59 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:28.001906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9882013-fb09-4eb5-9d8e-8e44f5d553af None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:28.002460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9882013-fb09-4eb5-9d8e-8e44f5d553af None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:28.002551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9882013-fb09-4eb5-9d8e-8e44f5d553af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:28.024765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e9882013-fb09-4eb5-9d8e-8e44f5d553af None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:28.026331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e9882013-fb09-4eb5-9d8e-8e44f5d553af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:28.094209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:28.095801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:28.103602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 976/1955] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:28.109790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6347c891-d522-4f65-a193-7fba767315fa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:28.114000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9882013-fb09-4eb5-9d8e-8e44f5d553af None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:28.115544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e9882013-fb09-4eb5-9d8e-8e44f5d553af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:28.121128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:28.122377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:28.122661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:28.122940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] PATH_INFO: `/v3/users/f79c5ac46cf941bfa3691be50fb1e34b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:28.124220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79c5ac46cf941bfa3691be50fb1e34b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:28.126321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:28.183586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79c5ac46cf941bfa3691be50fb1e34b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.193679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9882013-fb09-4eb5-9d8e-8e44f5d553af None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:28.194942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79c5ac46cf941bfa3691be50fb1e34b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.197804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d45db-6155-4517-ba17-0b87002eb30e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:28.198484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d45db-6155-4517-ba17-0b87002eb30e service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:28.198776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d45db-6155-4517-ba17-0b87002eb30e service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:28.199055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d45db-6155-4517-ba17-0b87002eb30e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:28.210309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79c5ac46cf941bfa3691be50fb1e34b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.210684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d45db-6155-4517-ba17-0b87002eb30e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:28.211177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79c5ac46cf941bfa3691be50fb1e34b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.211818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79c5ac46cf941bfa3691be50fb1e34b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.212008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d45db-6155-4517-ba17-0b87002eb30e service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:28.223801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 977/1956] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:20:28.230090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 980/1957] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:28 2022] DELETE /identity/v3/users/f79c5ac46cf941bfa3691be50fb1e34b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:28.235058 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a584d1-6d36-421b-afa7-c3007b3da3bc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:28.235324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a584d1-6d36-421b-afa7-c3007b3da3bc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:28.235599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a584d1-6d36-421b-afa7-c3007b3da3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:28.255771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-04a584d1-6d36-421b-afa7-c3007b3da3bc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:28.257024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-04a584d1-6d36-421b-afa7-c3007b3da3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:28.272750 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:28.335611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 978/1958] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:28.348493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:28.350356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:28.357519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:28.431254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ea167-4071-4970-9599-9ef6d9626f6b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:28.435133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1fe39-fcea-4156-ab73-1eda9c8af812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:28.435790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1fe39-fcea-4156-ab73-1eda9c8af812 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:28.436074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1fe39-fcea-4156-ab73-1eda9c8af812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:28.436348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1fe39-fcea-4156-ab73-1eda9c8af812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:28.449070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1fe39-fcea-4156-ab73-1eda9c8af812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:28.450600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1fe39-fcea-4156-ab73-1eda9c8af812 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:28.464533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 981/1959] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:28.545452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a584d1-6d36-421b-afa7-c3007b3da3bc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:28.547691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-04a584d1-6d36-421b-afa7-c3007b3da3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:28.562421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a584d1-6d36-421b-afa7-c3007b3da3bc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:28.568560 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:28.569554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:28.569947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:28.570464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] PATH_INFO: `/v3/projects/b7ce7b4c18964df0b1e8da760bf15e7a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:28.572152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ce7b4c18964df0b1e8da760bf15e7a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:28.574480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:28.633324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ce7b4c18964df0b1e8da760bf15e7a', 'request_id': 'req-0ab0c2ed-8403-4f33-9315-04f5365219c6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.645254 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ce7b4c18964df0b1e8da760bf15e7a', 'request_id': 'req-0ab0c2ed-8403-4f33-9315-04f5365219c6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.652450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ce7b4c18964df0b1e8da760bf15e7a', 'request_id': 'req-0ab0c2ed-8403-4f33-9315-04f5365219c6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.663743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] The token cache is being invalidate because project b7ce7b4c18964df0b1e8da760bf15e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:28.664028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.664684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.666497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 979/1960] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:28 2022] DELETE /identity/v3/projects/b7ce7b4c18964df0b1e8da760bf15e7a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:28.670494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1fe39-fcea-4156-ab73-1eda9c8af812 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:28.682388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1fe39-fcea-4156-ab73-1eda9c8af812 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:28.749101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:28.750252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:28.750419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:28.750899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] PATH_INFO: `/v3/users/cadedb16989543d4b9c929d7cb435585` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:28.752891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cadedb16989543d4b9c929d7cb435585)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:28.754456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:28.825169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cadedb16989543d4b9c929d7cb435585'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.836907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cadedb16989543d4b9c929d7cb435585'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.865508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cadedb16989543d4b9c929d7cb435585'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.866766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cadedb16989543d4b9c929d7cb435585'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.867687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-34a36f82-ddfb-4676-980d-7f70e7d60e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cadedb16989543d4b9c929d7cb435585'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:28.888785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:28.889375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 982/1961] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:28 2022] DELETE /identity/v3/users/cadedb16989543d4b9c929d7cb435585 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:28.890542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:28.911369 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0c2ed-8403-4f33-9315-04f5365219c6 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:28.987268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:28.988157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:28.988440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:28.989030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] PATH_INFO: `/v3/projects/cfe294a0659d4203971216d53f516dc2` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:28.990280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe294a0659d4203971216d53f516dc2)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:28.992686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.046406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:29.053079 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe294a0659d4203971216d53f516dc2', 'request_id': 'req-2ed845ba-4698-4712-82d4-ee50ef36629f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.064926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe294a0659d4203971216d53f516dc2', 'request_id': 'req-2ed845ba-4698-4712-82d4-ee50ef36629f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.074202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe294a0659d4203971216d53f516dc2', 'request_id': 'req-2ed845ba-4698-4712-82d4-ee50ef36629f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.087950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] The token cache is being invalidate because project cfe294a0659d4203971216d53f516dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:29.088242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.088956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ed845ba-4698-4712-82d4-ee50ef36629f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.090634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 980/1962] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:28 2022] DELETE /identity/v3/projects/cfe294a0659d4203971216d53f516dc2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:29.095626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f9e5-df21-4a5f-b074-d779916966cd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.095910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f9e5-df21-4a5f-b074-d779916966cd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.096284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205f9e5-df21-4a5f-b074-d779916966cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.119925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4205f9e5-df21-4a5f-b074-d779916966cd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:29.121166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4205f9e5-df21-4a5f-b074-d779916966cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:29.145713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:29.205082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 983/1963] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:29.214300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33941c-f2dd-4c43-939f-423600d5f884 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.214623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33941c-f2dd-4c43-939f-423600d5f884 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.214906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33941c-f2dd-4c43-939f-423600d5f884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.236027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3a33941c-f2dd-4c43-939f-423600d5f884 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:29.237323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3a33941c-f2dd-4c43-939f-423600d5f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:29.240140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f9e5-df21-4a5f-b074-d779916966cd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.241907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4205f9e5-df21-4a5f-b074-d779916966cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:29.249013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 981/1964] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:29.253330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33941c-f2dd-4c43-939f-423600d5f884 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.257242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205f9e5-df21-4a5f-b074-d779916966cd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.263339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33941c-f2dd-4c43-939f-423600d5f884 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.266657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.268128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.268551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.268839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] PATH_INFO: `/v3/projects/79a292322b8d471988ba77c75c7c4d53` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.269177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.269866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.270185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.270427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.270848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a292322b8d471988ba77c75c7c4d53)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.271171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:29.271490 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.273011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.273137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.288820 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 982/1965] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:20:29.292647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.305509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88cb9c5-b2aa-49cc-979e-0b623a9e1ca1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.311439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd3e2a-342f-4b7d-b115-ca364648deb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.312127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd3e2a-342f-4b7d-b115-ca364648deb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.312440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd3e2a-342f-4b7d-b115-ca364648deb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.312722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd3e2a-342f-4b7d-b115-ca364648deb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.313678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd3e2a-342f-4b7d-b115-ca364648deb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.316443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd3e2a-342f-4b7d-b115-ca364648deb1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.345193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a292322b8d471988ba77c75c7c4d53', 'request_id': 'req-0d5e2dd3-2a40-470a-be73-ec338ab55478'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.346242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 983/1966] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:29.350569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd3e2a-342f-4b7d-b115-ca364648deb1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.355017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a292322b8d471988ba77c75c7c4d53', 'request_id': 'req-0d5e2dd3-2a40-470a-be73-ec338ab55478'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.364461 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a292322b8d471988ba77c75c7c4d53', 'request_id': 'req-0d5e2dd3-2a40-470a-be73-ec338ab55478'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.376760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] The token cache is being invalidate because project 79a292322b8d471988ba77c75c7c4d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:29.377206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.377950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0d5e2dd3-2a40-470a-be73-ec338ab55478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:29.379892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 984/1967] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:29 2022] DELETE /identity/v3/projects/79a292322b8d471988ba77c75c7c4d53 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:29.384885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e52bf-7b73-41a2-b7fe-d31a855de3ff None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.385167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e52bf-7b73-41a2-b7fe-d31a855de3ff None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.385449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e52bf-7b73-41a2-b7fe-d31a855de3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.405407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1b8e52bf-7b73-41a2-b7fe-d31a855de3ff None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:29.406706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1b8e52bf-7b73-41a2-b7fe-d31a855de3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:29.430091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd3e2a-342f-4b7d-b115-ca364648deb1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.435757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.436456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.436743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.437024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.437937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.440186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.470622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 984/1968] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/users => generated 285 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:29.474809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.476575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:29.479501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 985/1969] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:29.489480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6d98-e7f8-465a-860a-2621100a8c82 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.489809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6d98-e7f8-465a-860a-2621100a8c82 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.489935 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57378dd-2b13-42dc-b8b5-150fe3e29769 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.490241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6d98-e7f8-465a-860a-2621100a8c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.495556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac6aa6-b46d-4c81-a620-80030326f18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.496249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac6aa6-b46d-4c81-a620-80030326f18d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.496571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac6aa6-b46d-4c81-a620-80030326f18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.496805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac6aa6-b46d-4c81-a620-80030326f18d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.497521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac6aa6-b46d-4c81-a620-80030326f18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.499207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac6aa6-b46d-4c81-a620-80030326f18d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.510101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e7bd6d98-e7f8-465a-860a-2621100a8c82 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:29.510439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 985/1970] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:29.511263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e7bd6d98-e7f8-465a-860a-2621100a8c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:29.514213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac6aa6-b46d-4c81-a620-80030326f18d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.519996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 986/1971] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:29.524425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd6d98-e7f8-465a-860a-2621100a8c82 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.525115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac6aa6-b46d-4c81-a620-80030326f18d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.531323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc20ad1-d1ad-4668-91a7-dac33c0710f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.532113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc20ad1-d1ad-4668-91a7-dac33c0710f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.532665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc20ad1-d1ad-4668-91a7-dac33c0710f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.533745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc20ad1-d1ad-4668-91a7-dac33c0710f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.534045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd6d98-e7f8-465a-860a-2621100a8c82 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.534356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc20ad1-d1ad-4668-91a7-dac33c0710f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.536172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc20ad1-d1ad-4668-91a7-dac33c0710f3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.539128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.539797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.540081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.540368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.541151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:29.541464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.543112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.549897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 986/1972] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:29.554773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc20ad1-d1ad-4668-91a7-dac33c0710f3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.559491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 987/1973] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:20:29.560983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.564966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc20ad1-d1ad-4668-91a7-dac33c0710f3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.570543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd6f682-7adb-49e3-a7cf-38e662fd0a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.571425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978cad9-7ad0-416a-9eeb-318e8d4161b8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.571561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6f682-7adb-49e3-a7cf-38e662fd0a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.571730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6f682-7adb-49e3-a7cf-38e662fd0a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.572013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd6f682-7adb-49e3-a7cf-38e662fd0a35 admin admin] PATH_INFO: `/v3/projects/fc310c704b1543be925d8764bac87320/users/73a0225f6d594018a19cec296937e2f8/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.576400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b6249-2568-41cc-b627-832e19926a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.577056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b6249-2568-41cc-b627-832e19926a5e admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.577336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b6249-2568-41cc-b627-832e19926a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.577606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b6249-2568-41cc-b627-832e19926a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.578334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b6249-2568-41cc-b627-832e19926a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.580973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b6249-2568-41cc-b627-832e19926a5e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.589642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6f682-7adb-49e3-a7cf-38e662fd0a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc310c704b1543be925d8764bac87320, user_id=73a0225f6d594018a19cec296937e2f8, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.592407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd6f682-7adb-49e3-a7cf-38e662fd0a35 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.614019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 987/1974] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:29 2022] PUT /identity/v3/projects/fc310c704b1543be925d8764bac87320/users/73a0225f6d594018a19cec296937e2f8/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:29.624406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 988/1975] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:29.624939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6e2b0-3dcd-4365-8a2a-4fb8e4de22ff None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.625316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6e2b0-3dcd-4365-8a2a-4fb8e4de22ff None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.625602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6e2b0-3dcd-4365-8a2a-4fb8e4de22ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.628934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b6249-2568-41cc-b627-832e19926a5e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.645089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:29.657074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8ea6e2b0-3dcd-4365-8a2a-4fb8e4de22ff None None] MFA Rules not processed for user `73a0225f6d594018a19cec296937e2f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:29.658557 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8ea6e2b0-3dcd-4365-8a2a-4fb8e4de22ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:29.681216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:29.706381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b6249-2568-41cc-b627-832e19926a5e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.713342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.714064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.714348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.714624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.715352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.717353 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.743844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 988/1976] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:29.744359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 989/1977] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:29.750758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6e2b0-3dcd-4365-8a2a-4fb8e4de22ff None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.756938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.759653 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:29.764894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6e2b0-3dcd-4365-8a2a-4fb8e4de22ff None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.770873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c619c-03e6-47ee-9be3-56f054112071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.771569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c619c-03e6-47ee-9be3-56f054112071 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.771854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c619c-03e6-47ee-9be3-56f054112071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.772132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c619c-03e6-47ee-9be3-56f054112071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.773718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c619c-03e6-47ee-9be3-56f054112071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.774587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262c619c-03e6-47ee-9be3-56f054112071 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.787081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 989/1978] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:29.793291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c619c-03e6-47ee-9be3-56f054112071 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.804646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c619c-03e6-47ee-9be3-56f054112071 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.810069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb287d-554f-4987-bdd9-36cc2900d031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.810758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb287d-554f-4987-bdd9-36cc2900d031 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.811049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb287d-554f-4987-bdd9-36cc2900d031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.811328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb287d-554f-4987-bdd9-36cc2900d031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.812061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb287d-554f-4987-bdd9-36cc2900d031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.813701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb287d-554f-4987-bdd9-36cc2900d031 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.834554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 990/1979] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:29.836192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb287d-554f-4987-bdd9-36cc2900d031 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:29.847093 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb287d-554f-4987-bdd9-36cc2900d031 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.852013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e487e6-d454-402f-9627-fb7e70d8edf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.852929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e487e6-d454-402f-9627-fb7e70d8edf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.853214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e487e6-d454-402f-9627-fb7e70d8edf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.853489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e487e6-d454-402f-9627-fb7e70d8edf8 admin admin] PATH_INFO: `/v3/projects/3fc6804e2bfb4277b1b9960b0c9d0565/users/14b3ec6be11140e491b1ff5cd4be80fb/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.864212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e4a70-a625-4355-88da-42759430a562 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:29.868601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae887e77-260e-427e-863f-f19e945342fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:29.869269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887e77-260e-427e-863f-f19e945342fa service glance] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.869555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887e77-260e-427e-863f-f19e945342fa service glance] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.869866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae887e77-260e-427e-863f-f19e945342fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.870946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e487e6-d454-402f-9627-fb7e70d8edf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc6804e2bfb4277b1b9960b0c9d0565, user_id=14b3ec6be11140e491b1ff5cd4be80fb, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.873771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e487e6-d454-402f-9627-fb7e70d8edf8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.880823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae887e77-260e-427e-863f-f19e945342fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:29.890448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae887e77-260e-427e-863f-f19e945342fa service glance] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:29.890591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 991/1980] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:29 2022] PUT /identity/v3/projects/3fc6804e2bfb4277b1b9960b0c9d0565/users/14b3ec6be11140e491b1ff5cd4be80fb/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:29.897280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 990/1981] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:20:29.901813 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00793e-638e-49ff-b310-fa150ea963a3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:29.902109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00793e-638e-49ff-b310-fa150ea963a3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:29.902392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00793e-638e-49ff-b310-fa150ea963a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:29.933904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8b00793e-638e-49ff-b310-fa150ea963a3 None None] MFA Rules not processed for user `14b3ec6be11140e491b1ff5cd4be80fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:29.935218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8b00793e-638e-49ff-b310-fa150ea963a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:30.010694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 992/1982] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:30.016520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae887e77-260e-427e-863f-f19e945342fa service glance] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.042141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae887e77-260e-427e-863f-f19e945342fa service glance] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.116914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240dae3-c183-4af1-933e-710f82a3737e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.117632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240dae3-c183-4af1-933e-710f82a3737e admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.117904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240dae3-c183-4af1-933e-710f82a3737e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.118181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240dae3-c183-4af1-933e-710f82a3737e admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.118904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240dae3-c183-4af1-933e-710f82a3737e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.121462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240dae3-c183-4af1-933e-710f82a3737e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.153518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 991/1983] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:30 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:30.157330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00793e-638e-49ff-b310-fa150ea963a3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.170426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00793e-638e-49ff-b310-fa150ea963a3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.258276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04665716-b13a-43c4-ae69-67b76c3bc2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.259081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04665716-b13a-43c4-ae69-67b76c3bc2a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.259366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04665716-b13a-43c4-ae69-67b76c3bc2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.259765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04665716-b13a-43c4-ae69-67b76c3bc2a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.261117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04665716-b13a-43c4-ae69-67b76c3bc2a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.263905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04665716-b13a-43c4-ae69-67b76c3bc2a3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.276314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:30.293722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 993/1984] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:30 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:30.298065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240dae3-c183-4af1-933e-710f82a3737e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.318375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240dae3-c183-4af1-933e-710f82a3737e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.323284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4699-8852-47db-866d-471c77f34643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.324197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4699-8852-47db-866d-471c77f34643 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.324454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4699-8852-47db-866d-471c77f34643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.324958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a4699-8852-47db-866d-471c77f34643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.325706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a4699-8852-47db-866d-471c77f34643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.329522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06a4699-8852-47db-866d-471c77f34643 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.332149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:30.347492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 992/1985] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:30.349506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04665716-b13a-43c4-ae69-67b76c3bc2a3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.360040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04665716-b13a-43c4-ae69-67b76c3bc2a3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.366045 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6751b0-5535-49c2-889c-82909ba602f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.366740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6751b0-5535-49c2-889c-82909ba602f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.367028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6751b0-5535-49c2-889c-82909ba602f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.367329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6751b0-5535-49c2-889c-82909ba602f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.368080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6751b0-5535-49c2-889c-82909ba602f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.369889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6751b0-5535-49c2-889c-82909ba602f9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.382288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 994/1986] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:30.386901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4699-8852-47db-866d-471c77f34643 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.398193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a4699-8852-47db-866d-471c77f34643 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.403298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effedbb-69ec-41e4-8189-b003d0be9314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.404252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effedbb-69ec-41e4-8189-b003d0be9314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.404536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effedbb-69ec-41e4-8189-b003d0be9314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.404820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effedbb-69ec-41e4-8189-b003d0be9314 admin admin] PATH_INFO: `/v3/projects/0b02d8c021544bb9a19f5f8de25438a1/users/a86af6e3cf2742dfa93e323e65697fca/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.422284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effedbb-69ec-41e4-8189-b003d0be9314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b02d8c021544bb9a19f5f8de25438a1, user_id=a86af6e3cf2742dfa93e323e65697fca, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.425058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effedbb-69ec-41e4-8189-b003d0be9314 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.447916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 993/1987] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:30 2022] PUT /identity/v3/projects/0b02d8c021544bb9a19f5f8de25438a1/users/a86af6e3cf2742dfa93e323e65697fca/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:30.459767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163afeef-e3a6-4103-8369-e4da89109811 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.460069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163afeef-e3a6-4103-8369-e4da89109811 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.460354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163afeef-e3a6-4103-8369-e4da89109811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.490330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-163afeef-e3a6-4103-8369-e4da89109811 None None] MFA Rules not processed for user `a86af6e3cf2742dfa93e323e65697fca`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:30.491583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-163afeef-e3a6-4103-8369-e4da89109811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:30.567767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 995/1988] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:30.572610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effedbb-69ec-41e4-8189-b003d0be9314 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.645425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effedbb-69ec-41e4-8189-b003d0be9314 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.650345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226f138-9e89-426b-aa7c-9808ccb2930d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.651022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226f138-9e89-426b-aa7c-9808ccb2930d admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.651303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226f138-9e89-426b-aa7c-9808ccb2930d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.651578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226f138-9e89-426b-aa7c-9808ccb2930d admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.652345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226f138-9e89-426b-aa7c-9808ccb2930d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.654380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5226f138-9e89-426b-aa7c-9808ccb2930d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.683187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 994/1989] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:30 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:30.687630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163afeef-e3a6-4103-8369-e4da89109811 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.699923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163afeef-e3a6-4103-8369-e4da89109811 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.768029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bbaf7-3219-4515-96ff-8dc2a23240c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.768723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bbaf7-3219-4515-96ff-8dc2a23240c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.769005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bbaf7-3219-4515-96ff-8dc2a23240c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.769277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01bbaf7-3219-4515-96ff-8dc2a23240c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.770098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bbaf7-3219-4515-96ff-8dc2a23240c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.771988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01bbaf7-3219-4515-96ff-8dc2a23240c3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.793345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 996/1990] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:30 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:30.797212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226f138-9e89-426b-aa7c-9808ccb2930d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.807633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226f138-9e89-426b-aa7c-9808ccb2930d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.817840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b2fc3-4ec9-403b-9186-e534cd8f6fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.818884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2fc3-4ec9-403b-9186-e534cd8f6fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.819434 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2fc3-4ec9-403b-9186-e534cd8f6fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.819917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b2fc3-4ec9-403b-9186-e534cd8f6fe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.820868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b2fc3-4ec9-403b-9186-e534cd8f6fe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.822614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b2fc3-4ec9-403b-9186-e534cd8f6fe9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.836408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 995/1991] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:30.840741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bbaf7-3219-4515-96ff-8dc2a23240c3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.856266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01bbaf7-3219-4515-96ff-8dc2a23240c3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.862224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc548-7c97-447b-94d7-f0503fc9795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.863130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc548-7c97-447b-94d7-f0503fc9795d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.863421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc548-7c97-447b-94d7-f0503fc9795d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.864003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5cc548-7c97-447b-94d7-f0503fc9795d admin admin] PATH_INFO: `/v3/projects/cb2ece8d326842c2988d0173dd17e54c/users/0281ffc715dd4bb981ac8c8925962aa5/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.881072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cc548-7c97-447b-94d7-f0503fc9795d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2ece8d326842c2988d0173dd17e54c, user_id=0281ffc715dd4bb981ac8c8925962aa5, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.883809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5cc548-7c97-447b-94d7-f0503fc9795d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:30.900025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 997/1992] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:30 2022] PUT /identity/v3/projects/cb2ece8d326842c2988d0173dd17e54c/users/0281ffc715dd4bb981ac8c8925962aa5/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:30.904147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b2fc3-4ec9-403b-9186-e534cd8f6fe9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:30.917731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b2fc3-4ec9-403b-9186-e534cd8f6fe9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:30.985349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5e11f-440c-49f5-b021-937b5528ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:30.986751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5e11f-440c-49f5-b021-937b5528ea3a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:30.986751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5e11f-440c-49f5-b021-937b5528ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:30.986751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5e11f-440c-49f5-b021-937b5528ea3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:30.987457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5e11f-440c-49f5-b021-937b5528ea3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:30.989522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5e11f-440c-49f5-b021-937b5528ea3a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.006308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 996/1993] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:31.007865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc548-7c97-447b-94d7-f0503fc9795d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.021180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5cc548-7c97-447b-94d7-f0503fc9795d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.026649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.027648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.028010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.028354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] PATH_INFO: `/v3/projects/cb2ece8d326842c2988d0173dd17e54c/users/0281ffc715dd4bb981ac8c8925962aa5/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.030716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2ece8d326842c2988d0173dd17e54c, user_id=0281ffc715dd4bb981ac8c8925962aa5, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.034379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.050561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 998/1994] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:31 2022] PUT /identity/v3/projects/cb2ece8d326842c2988d0173dd17e54c/users/0281ffc715dd4bb981ac8c8925962aa5/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:31.062484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1885a-541e-4e01-8e80-935fe5cf6073 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.063010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1885a-541e-4e01-8e80-935fe5cf6073 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.063420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1885a-541e-4e01-8e80-935fe5cf6073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.093213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7df1885a-541e-4e01-8e80-935fe5cf6073 None None] MFA Rules not processed for user `0281ffc715dd4bb981ac8c8925962aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:31.094640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7df1885a-541e-4e01-8e80-935fe5cf6073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:31.132326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:31.177988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 997/1995] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:31.190886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.192659 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:31.223961 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:31.256042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e943-9032-49d4-8135-9a06f550d7fe admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.259866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e46e-855f-4760-84d6-2447ff5f76e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.260613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e46e-855f-4760-84d6-2447ff5f76e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.260876 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e46e-855f-4760-84d6-2447ff5f76e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.261150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e46e-855f-4760-84d6-2447ff5f76e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.272724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526e46e-855f-4760-84d6-2447ff5f76e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.274145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526e46e-855f-4760-84d6-2447ff5f76e0 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.285094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 999/1996] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:31.328171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1885a-541e-4e01-8e80-935fe5cf6073 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:31.331443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7df1885a-541e-4e01-8e80-935fe5cf6073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:31.350166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1885a-541e-4e01-8e80-935fe5cf6073 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:31.354966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028b247-f40e-409e-a432-a3293e3f0317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:31.355637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028b247-f40e-409e-a432-a3293e3f0317 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:31.355918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028b247-f40e-409e-a432-a3293e3f0317 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:31.356189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028b247-f40e-409e-a432-a3293e3f0317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:31.435245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028b247-f40e-409e-a432-a3293e3f0317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:31.436951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028b247-f40e-409e-a432-a3293e3f0317 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:31.447829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 998/1997] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:34.512524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e46e-855f-4760-84d6-2447ff5f76e0 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:34.581381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e46e-855f-4760-84d6-2447ff5f76e0 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:34.586461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:34.587269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:34.587554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:34.587831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] PATH_INFO: `/v3/users/14b3ec6be11140e491b1ff5cd4be80fb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:34.589084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14b3ec6be11140e491b1ff5cd4be80fb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:34.591159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:34.664833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14b3ec6be11140e491b1ff5cd4be80fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:34.676088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14b3ec6be11140e491b1ff5cd4be80fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:34.694472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14b3ec6be11140e491b1ff5cd4be80fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:34.695191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14b3ec6be11140e491b1ff5cd4be80fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:34.695819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14b3ec6be11140e491b1ff5cd4be80fb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:34.713423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1000/1998] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:34 2022] DELETE /identity/v3/users/14b3ec6be11140e491b1ff5cd4be80fb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:34.715354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf214f1-abca-4b31-8874-18b562282477 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:34.715924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf214f1-abca-4b31-8874-18b562282477 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:34.716022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf214f1-abca-4b31-8874-18b562282477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:34.744433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0bf214f1-abca-4b31-8874-18b562282477 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:34.745693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0bf214f1-abca-4b31-8874-18b562282477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:34.815911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 999/1999] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:34.827077 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:34.831700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:34.898283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:34.909933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337afb0-2be1-458c-8196-f1e2abb8bec1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:34.916540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087a68a-82d8-45ad-9bb9-32384591f507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:34.916540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087a68a-82d8-45ad-9bb9-32384591f507 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:34.917079 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087a68a-82d8-45ad-9bb9-32384591f507 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:34.917415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087a68a-82d8-45ad-9bb9-32384591f507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:34.928287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087a68a-82d8-45ad-9bb9-32384591f507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:34.929799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087a68a-82d8-45ad-9bb9-32384591f507 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:34.940967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1001/2000] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:20:35.239661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf214f1-abca-4b31-8874-18b562282477 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:35.241162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0bf214f1-abca-4b31-8874-18b562282477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:35.263299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf214f1-abca-4b31-8874-18b562282477 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:35.268259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:35.269340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:35.269628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:35.269940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] PATH_INFO: `/v3/projects/3fc6804e2bfb4277b1b9960b0c9d0565` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:35.271257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fc6804e2bfb4277b1b9960b0c9d0565)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:35.273173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:35.330733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fc6804e2bfb4277b1b9960b0c9d0565', 'request_id': 'req-124a2784-f936-48be-bd92-b8a53974110c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.338004 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fc6804e2bfb4277b1b9960b0c9d0565', 'request_id': 'req-124a2784-f936-48be-bd92-b8a53974110c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.348518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fc6804e2bfb4277b1b9960b0c9d0565', 'request_id': 'req-124a2784-f936-48be-bd92-b8a53974110c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.349954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:35.365050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] The token cache is being invalidate because project 3fc6804e2bfb4277b1b9960b0c9d0565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:35.365394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.366121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.368300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1000/2001] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:35 2022] DELETE /identity/v3/projects/3fc6804e2bfb4277b1b9960b0c9d0565 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:35.373329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087a68a-82d8-45ad-9bb9-32384591f507 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:35.383889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087a68a-82d8-45ad-9bb9-32384591f507 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:35.454777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:35.456042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:35.456042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:35.456319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] PATH_INFO: `/v3/users/a86af6e3cf2742dfa93e323e65697fca` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:35.457589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86af6e3cf2742dfa93e323e65697fca)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:35.459583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:35.518536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86af6e3cf2742dfa93e323e65697fca'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.528696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86af6e3cf2742dfa93e323e65697fca'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.550359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86af6e3cf2742dfa93e323e65697fca'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.551203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86af6e3cf2742dfa93e323e65697fca'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.551918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86af6e3cf2742dfa93e323e65697fca'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:35.565853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1002/2002] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:35 2022] DELETE /identity/v3/users/a86af6e3cf2742dfa93e323e65697fca => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:35.875031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:35.876721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:35.891906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a2784-f936-48be-bd92-b8a53974110c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:35.957058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:35.958068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:35.958350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:35.958626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] PATH_INFO: `/v3/projects/0b02d8c021544bb9a19f5f8de25438a1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:35.959830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b02d8c021544bb9a19f5f8de25438a1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:35.961869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:36.016103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b02d8c021544bb9a19f5f8de25438a1', 'request_id': 'req-cd1a0ee5-33ac-4401-9370-b99357b43290'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.023829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b02d8c021544bb9a19f5f8de25438a1', 'request_id': 'req-cd1a0ee5-33ac-4401-9370-b99357b43290'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.046679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b02d8c021544bb9a19f5f8de25438a1', 'request_id': 'req-cd1a0ee5-33ac-4401-9370-b99357b43290'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.060494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] The token cache is being invalidate because project 0b02d8c021544bb9a19f5f8de25438a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:36.060938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.062108 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.063772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1001/2003] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:35 2022] DELETE /identity/v3/projects/0b02d8c021544bb9a19f5f8de25438a1 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:36.072774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:36.074332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:36.087414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0edc7e5-ab5f-48f5-b531-a038aa242915 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:36.149383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:36.150364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:36.150646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:36.150930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] PATH_INFO: `/v3/users/0281ffc715dd4bb981ac8c8925962aa5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:36.152208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0281ffc715dd4bb981ac8c8925962aa5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:36.154269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:36.174882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:36.213067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0281ffc715dd4bb981ac8c8925962aa5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.221462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0281ffc715dd4bb981ac8c8925962aa5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.234941 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0281ffc715dd4bb981ac8c8925962aa5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.235754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0281ffc715dd4bb981ac8c8925962aa5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.237438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0e098656-17af-4415-8fd4-a411b3028bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0281ffc715dd4bb981ac8c8925962aa5'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:36.252411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1003/2004] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:36 2022] DELETE /identity/v3/users/0281ffc715dd4bb981ac8c8925962aa5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:36.821636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:36.823202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:36.845895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0ee5-33ac-4401-9370-b99357b43290 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:36.939971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:36.940924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:36.941209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:36.941484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] PATH_INFO: `/v3/projects/cb2ece8d326842c2988d0173dd17e54c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:36.942783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2ece8d326842c2988d0173dd17e54c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:36.944826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:36.969386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:37.004022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2ece8d326842c2988d0173dd17e54c', 'request_id': 'req-79624cf9-c681-4bdb-a225-3ee1feefacab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.010371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2ece8d326842c2988d0173dd17e54c', 'request_id': 'req-79624cf9-c681-4bdb-a225-3ee1feefacab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.018907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2ece8d326842c2988d0173dd17e54c', 'request_id': 'req-79624cf9-c681-4bdb-a225-3ee1feefacab'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.031254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] The token cache is being invalidate because project cb2ece8d326842c2988d0173dd17e54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:37.031708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.032915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-79624cf9-c681-4bdb-a225-3ee1feefacab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:37.035275 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1002/2005] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:36 2022] DELETE /identity/v3/projects/cb2ece8d326842c2988d0173dd17e54c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:37.041437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1eb57-8177-429a-89e0-3a32820dbe6f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.042067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1eb57-8177-429a-89e0-3a32820dbe6f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.042067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1eb57-8177-429a-89e0-3a32820dbe6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.061204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-85e1eb57-8177-429a-89e0-3a32820dbe6f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:37.062633 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-85e1eb57-8177-429a-89e0-3a32820dbe6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:37.138336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1004/2006] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:37.149302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446f29-ff29-4979-a0d3-ef193e771cde None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.149607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446f29-ff29-4979-a0d3-ef193e771cde None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.149910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09446f29-ff29-4979-a0d3-ef193e771cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.170319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-09446f29-ff29-4979-a0d3-ef193e771cde None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:37.171925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-09446f29-ff29-4979-a0d3-ef193e771cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:37.183648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1003/2007] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:37.188123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1eb57-8177-429a-89e0-3a32820dbe6f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.199480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1eb57-8177-429a-89e0-3a32820dbe6f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.207712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.209424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.209841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.210099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.210820 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:37.211124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.212982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.226582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1005/2008] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:37.231983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446f29-ff29-4979-a0d3-ef193e771cde None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.246295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09446f29-ff29-4979-a0d3-ef193e771cde None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.252017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63edc72-8b7a-4cdb-ae5f-be4900d8b897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.252963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63edc72-8b7a-4cdb-ae5f-be4900d8b897 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.253253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63edc72-8b7a-4cdb-ae5f-be4900d8b897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.253574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63edc72-8b7a-4cdb-ae5f-be4900d8b897 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.254476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63edc72-8b7a-4cdb-ae5f-be4900d8b897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.256386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63edc72-8b7a-4cdb-ae5f-be4900d8b897 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.287067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1004/2009] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:37 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:37.291388 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.304208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1b4b6-5b64-4da9-b436-eef0542908dd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.379660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52c44e-6cb8-4b7c-9341-72d8cd5dca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.380372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52c44e-6cb8-4b7c-9341-72d8cd5dca05 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.381039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52c44e-6cb8-4b7c-9341-72d8cd5dca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.382668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52c44e-6cb8-4b7c-9341-72d8cd5dca05 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.383428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52c44e-6cb8-4b7c-9341-72d8cd5dca05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.385422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52c44e-6cb8-4b7c-9341-72d8cd5dca05 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.415494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1006/2010] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:37 2022] POST /identity/v3/users => generated 284 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:37.419633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63edc72-8b7a-4cdb-ae5f-be4900d8b897 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.431855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63edc72-8b7a-4cdb-ae5f-be4900d8b897 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.436742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14aa08-13d4-4d37-8dd3-48faa9f3d0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.437439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14aa08-13d4-4d37-8dd3-48faa9f3d0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.437750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14aa08-13d4-4d37-8dd3-48faa9f3d0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.438019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14aa08-13d4-4d37-8dd3-48faa9f3d0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.438863 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14aa08-13d4-4d37-8dd3-48faa9f3d0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.440600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14aa08-13d4-4d37-8dd3-48faa9f3d0ad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.457105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1005/2011] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:37.458797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52c44e-6cb8-4b7c-9341-72d8cd5dca05 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.480502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52c44e-6cb8-4b7c-9341-72d8cd5dca05 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.489514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d98980-4ad1-4ed9-9b73-84a3bc5f68ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.491784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d98980-4ad1-4ed9-9b73-84a3bc5f68ec admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.491784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d98980-4ad1-4ed9-9b73-84a3bc5f68ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.492254 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d98980-4ad1-4ed9-9b73-84a3bc5f68ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.493020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d98980-4ad1-4ed9-9b73-84a3bc5f68ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.494752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d98980-4ad1-4ed9-9b73-84a3bc5f68ec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.500806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:37.508828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1007/2012] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:37.513262 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14aa08-13d4-4d37-8dd3-48faa9f3d0ad admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.525509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14aa08-13d4-4d37-8dd3-48faa9f3d0ad admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.532463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dfe9d-332b-45d4-8193-92008f6856c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.534703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dfe9d-332b-45d4-8193-92008f6856c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.534899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dfe9d-332b-45d4-8193-92008f6856c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.535166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868dfe9d-332b-45d4-8193-92008f6856c3 admin admin] PATH_INFO: `/v3/projects/0e7c99fbac26478a813421223e60bef9/users/ef82c6695c63440abbb336560efafa40/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.554196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dfe9d-332b-45d4-8193-92008f6856c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e7c99fbac26478a813421223e60bef9, user_id=ef82c6695c63440abbb336560efafa40, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.556911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868dfe9d-332b-45d4-8193-92008f6856c3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.579635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1006/2013] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:37 2022] PUT /identity/v3/projects/0e7c99fbac26478a813421223e60bef9/users/ef82c6695c63440abbb336560efafa40/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:37.602108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632d88-a2c1-4cee-932f-20be3a31e7d3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.602425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632d88-a2c1-4cee-932f-20be3a31e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.602688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75632d88-a2c1-4cee-932f-20be3a31e7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.638729 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-75632d88-a2c1-4cee-932f-20be3a31e7d3 None None] MFA Rules not processed for user `ef82c6695c63440abbb336560efafa40`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:37.639963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-75632d88-a2c1-4cee-932f-20be3a31e7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:37.729937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1008/2014] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:37.734409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dfe9d-332b-45d4-8193-92008f6856c3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.818726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868dfe9d-332b-45d4-8193-92008f6856c3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.823690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca824-3093-4c67-bcef-41da8273f136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.824278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca824-3093-4c67-bcef-41da8273f136 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.824565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca824-3093-4c67-bcef-41da8273f136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.824843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ca824-3093-4c67-bcef-41da8273f136 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.825567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ca824-3093-4c67-bcef-41da8273f136 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.827573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ca824-3093-4c67-bcef-41da8273f136 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.860343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1007/2015] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:37 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:37.862210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75632d88-a2c1-4cee-932f-20be3a31e7d3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:37.882494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75632d88-a2c1-4cee-932f-20be3a31e7d3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:37.954916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f30f-f70b-4c66-93f3-2da059d20aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:37.955892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f30f-f70b-4c66-93f3-2da059d20aad admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:37.956241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f30f-f70b-4c66-93f3-2da059d20aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:37.956580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0f30f-f70b-4c66-93f3-2da059d20aad admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:37.957474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0f30f-f70b-4c66-93f3-2da059d20aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:37.959554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0f30f-f70b-4c66-93f3-2da059d20aad admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:37.988400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1009/2016] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:37 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:37.993159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca824-3093-4c67-bcef-41da8273f136 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.005935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ca824-3093-4c67-bcef-41da8273f136 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.011137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1837d-5067-470e-8d6b-89823217c1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.011826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1837d-5067-470e-8d6b-89823217c1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.012112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1837d-5067-470e-8d6b-89823217c1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.012390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1837d-5067-470e-8d6b-89823217c1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.013122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1837d-5067-470e-8d6b-89823217c1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.014804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1837d-5067-470e-8d6b-89823217c1c8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.027583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1008/2017] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:38.034085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f30f-f70b-4c66-93f3-2da059d20aad admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.052486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0f30f-f70b-4c66-93f3-2da059d20aad admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.063217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55304c5c-bced-4caa-93b4-5fe35c1bb207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.064505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55304c5c-bced-4caa-93b4-5fe35c1bb207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.064938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55304c5c-bced-4caa-93b4-5fe35c1bb207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.065289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55304c5c-bced-4caa-93b4-5fe35c1bb207 admin admin] PATH_INFO: `/v3/projects/cc1aafa6c46d4a9abe69277c682e00d8/users/f886847b45aa4b94b82f510160c8ce50/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.086027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55304c5c-bced-4caa-93b4-5fe35c1bb207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1aafa6c46d4a9abe69277c682e00d8, user_id=f886847b45aa4b94b82f510160c8ce50, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.088778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55304c5c-bced-4caa-93b4-5fe35c1bb207 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.117997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1010/2018] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:38 2022] PUT /identity/v3/projects/cc1aafa6c46d4a9abe69277c682e00d8/users/f886847b45aa4b94b82f510160c8ce50/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:38.118788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1837d-5067-470e-8d6b-89823217c1c8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.132834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1837d-5067-470e-8d6b-89823217c1c8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.171456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:38.224545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d963cd-706e-4fdc-92b9-978fd3087c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.225472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d963cd-706e-4fdc-92b9-978fd3087c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.225775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d963cd-706e-4fdc-92b9-978fd3087c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.226061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d963cd-706e-4fdc-92b9-978fd3087c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.227031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d963cd-706e-4fdc-92b9-978fd3087c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.228716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d963cd-706e-4fdc-92b9-978fd3087c59 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.244717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1009/2019] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:38.246417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55304c5c-bced-4caa-93b4-5fe35c1bb207 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.260040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55304c5c-bced-4caa-93b4-5fe35c1bb207 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.264994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.265946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.266235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.266518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] PATH_INFO: `/v3/projects/cc1aafa6c46d4a9abe69277c682e00d8/users/f886847b45aa4b94b82f510160c8ce50/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.268969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1aafa6c46d4a9abe69277c682e00d8, user_id=f886847b45aa4b94b82f510160c8ce50, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.272151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.293167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1011/2020] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:38 2022] PUT /identity/v3/projects/cc1aafa6c46d4a9abe69277c682e00d8/users/f886847b45aa4b94b82f510160c8ce50/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:38.295086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b127-04f1-47a7-bc61-ceb00dd801df None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.295374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b127-04f1-47a7-bc61-ceb00dd801df None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.295656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b127-04f1-47a7-bc61-ceb00dd801df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.340093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-88e9b127-04f1-47a7-bc61-ceb00dd801df None None] MFA Rules not processed for user `f886847b45aa4b94b82f510160c8ce50`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:38.342742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-88e9b127-04f1-47a7-bc61-ceb00dd801df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:38.479299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1010/2021] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:38.494688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.501894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:38.548421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:38.575092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6dc07a-278c-4cc8-acf8-3fd4ebbc83f9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.580825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.581514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.581861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.582148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.594488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.596275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.624529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1012/2022] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:38.751602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9b127-04f1-47a7-bc61-ceb00dd801df None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:38.762016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9b127-04f1-47a7-bc61-ceb00dd801df None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:38.766496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b21b03-28d9-4a8f-9222-d42c6d1db1d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:38.767350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b21b03-28d9-4a8f-9222-d42c6d1db1d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:38.767733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b21b03-28d9-4a8f-9222-d42c6d1db1d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:38.768078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b21b03-28d9-4a8f-9222-d42c6d1db1d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:38.831159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b21b03-28d9-4a8f-9222-d42c6d1db1d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:38.832641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b21b03-28d9-4a8f-9222-d42c6d1db1d9 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:38.847700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1011/2023] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:40.145029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:40.147491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:40.218099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd917-78ae-4f64-a32f-6e332768093e service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:40.223914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:40.224979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:40.225831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:40.225831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] PATH_INFO: `/v3/users/2e3c7eb6a5cb4515b6f48520535fea76` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:40.226854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3c7eb6a5cb4515b6f48520535fea76)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:40.228953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:40.297026 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3c7eb6a5cb4515b6f48520535fea76'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.308370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3c7eb6a5cb4515b6f48520535fea76'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.323074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3c7eb6a5cb4515b6f48520535fea76'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.324209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3c7eb6a5cb4515b6f48520535fea76'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.325563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-84341f58-084d-4b18-b8cf-77158dfed6f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3c7eb6a5cb4515b6f48520535fea76'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.340963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1013/2024] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:40 2022] DELETE /identity/v3/users/2e3c7eb6a5cb4515b6f48520535fea76 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:40.585098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b21b03-28d9-4a8f-9222-d42c6d1db1d9 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:40.597593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b21b03-28d9-4a8f-9222-d42c6d1db1d9 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:40.665685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:40.688276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:40.689494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:40.689979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:40.690562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] PATH_INFO: `/v3/projects/8788b8d25f234df6895ea3f8fa046610` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:40.691677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8788b8d25f234df6895ea3f8fa046610)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:40.693758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:40.751603 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8788b8d25f234df6895ea3f8fa046610', 'request_id': 'req-3b6d0613-eef3-4017-b979-f01e87326aeb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.759405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8788b8d25f234df6895ea3f8fa046610', 'request_id': 'req-3b6d0613-eef3-4017-b979-f01e87326aeb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.781594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8788b8d25f234df6895ea3f8fa046610', 'request_id': 'req-3b6d0613-eef3-4017-b979-f01e87326aeb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.795403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] The token cache is being invalidate because project 8788b8d25f234df6895ea3f8fa046610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:40.795729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.796748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3b6d0613-eef3-4017-b979-f01e87326aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:40.799039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1012/2025] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:40 2022] DELETE /identity/v3/projects/8788b8d25f234df6895ea3f8fa046610 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:40.804098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61859ea-f4b0-4c64-b576-f14060779646 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:40.804395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61859ea-f4b0-4c64-b576-f14060779646 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:40.804703 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61859ea-f4b0-4c64-b576-f14060779646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:40.824075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f61859ea-f4b0-4c64-b576-f14060779646 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:40.825589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f61859ea-f4b0-4c64-b576-f14060779646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:40.883018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:40.898897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1014/2026] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:40.907913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a3e3-d918-43f8-a1e5-1fcb5b409625 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:40.908547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a3e3-d918-43f8-a1e5-1fcb5b409625 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:40.909338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a3e3-d918-43f8-a1e5-1fcb5b409625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:40.929912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9c58a3e3-d918-43f8-a1e5-1fcb5b409625 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:40.931061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9c58a3e3-d918-43f8-a1e5-1fcb5b409625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:40.940958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1013/2027] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:40.945967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61859ea-f4b0-4c64-b576-f14060779646 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:40.959587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61859ea-f4b0-4c64-b576-f14060779646 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:40.964644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:40.965328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:40.965614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:40.965904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:40.966665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:40.967090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:40.968803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:40.983862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1015/2028] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:40.989377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58a3e3-d918-43f8-a1e5-1fcb5b409625 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:40.999204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58a3e3-d918-43f8-a1e5-1fcb5b409625 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:41.006453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554859d-6bd4-403a-8f6b-0a7752444292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:41.007122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554859d-6bd4-403a-8f6b-0a7752444292 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:41.007392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554859d-6bd4-403a-8f6b-0a7752444292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:41.007668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554859d-6bd4-403a-8f6b-0a7752444292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:41.008427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554859d-6bd4-403a-8f6b-0a7752444292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:41.010407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554859d-6bd4-403a-8f6b-0a7752444292 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:41.038488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1014/2029] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:40 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:41.042695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:41.053615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57704c86-4ce2-4c70-9785-787f2cf87bfd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:41.122289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45526ec7-3f85-4cb6-879f-d96a282a25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:41.122289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45526ec7-3f85-4cb6-879f-d96a282a25aa admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:41.122669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45526ec7-3f85-4cb6-879f-d96a282a25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:41.122906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45526ec7-3f85-4cb6-879f-d96a282a25aa admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:41.123621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45526ec7-3f85-4cb6-879f-d96a282a25aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:41.125559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45526ec7-3f85-4cb6-879f-d96a282a25aa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:41.149275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1016/2030] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:41 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:41.153596 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554859d-6bd4-403a-8f6b-0a7752444292 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:41.165998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554859d-6bd4-403a-8f6b-0a7752444292 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:41.172167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dab4ff-0302-4f6e-b843-6932c7c7b804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:41.172839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dab4ff-0302-4f6e-b843-6932c7c7b804 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:41.174700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dab4ff-0302-4f6e-b843-6932c7c7b804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:41.174991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dab4ff-0302-4f6e-b843-6932c7c7b804 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:41.175729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dab4ff-0302-4f6e-b843-6932c7c7b804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:41.177356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dab4ff-0302-4f6e-b843-6932c7c7b804 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:41.189581 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1015/2031] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:41.193140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45526ec7-3f85-4cb6-879f-d96a282a25aa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:41.203715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45526ec7-3f85-4cb6-879f-d96a282a25aa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:41.209530 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6252186-ec92-4c77-bf72-aca763c32bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:41.210241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6252186-ec92-4c77-bf72-aca763c32bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:41.210524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6252186-ec92-4c77-bf72-aca763c32bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:41.210860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6252186-ec92-4c77-bf72-aca763c32bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:41.211600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6252186-ec92-4c77-bf72-aca763c32bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:41.213221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6252186-ec92-4c77-bf72-aca763c32bfa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:41.225273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1017/2032] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:41.229407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dab4ff-0302-4f6e-b843-6932c7c7b804 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:41.240752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dab4ff-0302-4f6e-b843-6932c7c7b804 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:41.246649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:41.247538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:41.247821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:41.248101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] PATH_INFO: `/v3/projects/dd6d368e19c740859b337513799b07e9/users/8dd275728e7a4a5b9ff524d122570c45/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:41.265024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6d368e19c740859b337513799b07e9, user_id=8dd275728e7a4a5b9ff524d122570c45, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:41.268176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:41.284022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1016/2033] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:41 2022] PUT /identity/v3/projects/dd6d368e19c740859b337513799b07e9/users/8dd275728e7a4a5b9ff524d122570c45/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:41.290399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66449bf7-1328-4e35-b3ee-9924a5267f7c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:41.290870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66449bf7-1328-4e35-b3ee-9924a5267f7c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:41.291338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66449bf7-1328-4e35-b3ee-9924a5267f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:41.317340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-66449bf7-1328-4e35-b3ee-9924a5267f7c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:41.318619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-66449bf7-1328-4e35-b3ee-9924a5267f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:41.384726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1018/2034] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:41.394735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:41.396934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:41.458940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7809b5-0fc6-4009-b51d-ff2be471c92f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:41.462920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddca32-cc52-4309-9ccc-7f3f0e2c91f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:41.463547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddca32-cc52-4309-9ccc-7f3f0e2c91f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:41.463833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddca32-cc52-4309-9ccc-7f3f0e2c91f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:41.464197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ddca32-cc52-4309-9ccc-7f3f0e2c91f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:41.476058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddca32-cc52-4309-9ccc-7f3f0e2c91f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:41.477582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ddca32-cc52-4309-9ccc-7f3f0e2c91f4 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:41.489770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1017/2035] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:42.348011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be75fe7-0f87-40c2-a08e-4731029fda05 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:42.348582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be75fe7-0f87-40c2-a08e-4731029fda05 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:42.348904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be75fe7-0f87-40c2-a08e-4731029fda05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:42.380955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1be75fe7-0f87-40c2-a08e-4731029fda05 None None] MFA Rules not processed for user `8dd275728e7a4a5b9ff524d122570c45`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:42.382381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1be75fe7-0f87-40c2-a08e-4731029fda05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:42.462849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:42.478315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1019/2036] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:42.479426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddca32-cc52-4309-9ccc-7f3f0e2c91f4 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:42.492171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ddca32-cc52-4309-9ccc-7f3f0e2c91f4 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:42.497518 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6752d2-14cb-4154-a49e-a021d13bbd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:42.498463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6752d2-14cb-4154-a49e-a021d13bbd69 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:42.498751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6752d2-14cb-4154-a49e-a021d13bbd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:42.499086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6752d2-14cb-4154-a49e-a021d13bbd69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:42.499854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6752d2-14cb-4154-a49e-a021d13bbd69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:42.502100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6752d2-14cb-4154-a49e-a021d13bbd69 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:42.507226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:42.534863 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1018/2037] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:42 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:42.539325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be75fe7-0f87-40c2-a08e-4731029fda05 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:42.551630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be75fe7-0f87-40c2-a08e-4731029fda05 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:42.620471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ba03-b485-4d27-b62b-e7ec3d0ffe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:42.621212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ba03-b485-4d27-b62b-e7ec3d0ffe14 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:42.621496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ba03-b485-4d27-b62b-e7ec3d0ffe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:42.621814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ba03-b485-4d27-b62b-e7ec3d0ffe14 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:42.622523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6ba03-b485-4d27-b62b-e7ec3d0ffe14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:42.624517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6ba03-b485-4d27-b62b-e7ec3d0ffe14 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:42.652311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1020/2038] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:42 2022] POST /identity/v3/users => generated 309 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:42.656515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6752d2-14cb-4154-a49e-a021d13bbd69 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:42.667763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6752d2-14cb-4154-a49e-a021d13bbd69 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:42.672731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd87172-2aba-4acd-936f-bfed72f043e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:42.673470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd87172-2aba-4acd-936f-bfed72f043e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:42.673800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd87172-2aba-4acd-936f-bfed72f043e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:42.674196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd87172-2aba-4acd-936f-bfed72f043e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:42.674887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd87172-2aba-4acd-936f-bfed72f043e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:42.676531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd87172-2aba-4acd-936f-bfed72f043e5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:42.689697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1019/2039] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:42.694039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ba03-b485-4d27-b62b-e7ec3d0ffe14 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:42.704491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ba03-b485-4d27-b62b-e7ec3d0ffe14 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:42.710839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12372497-ff5a-43ca-9f07-8183c7c9b65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:42.711865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372497-ff5a-43ca-9f07-8183c7c9b65c admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:42.714439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372497-ff5a-43ca-9f07-8183c7c9b65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:42.714820 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12372497-ff5a-43ca-9f07-8183c7c9b65c admin admin] PATH_INFO: `/v3/projects/1e0ce8a87599448aba3c20e258c0eb50/users/29e612e6e6234fbda52532b6dbdba640/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:42.735279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12372497-ff5a-43ca-9f07-8183c7c9b65c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0ce8a87599448aba3c20e258c0eb50, user_id=29e612e6e6234fbda52532b6dbdba640, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:42.738107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12372497-ff5a-43ca-9f07-8183c7c9b65c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:42.754311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1021/2040] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:42 2022] PUT /identity/v3/projects/1e0ce8a87599448aba3c20e258c0eb50/users/29e612e6e6234fbda52532b6dbdba640/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:42.758465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd87172-2aba-4acd-936f-bfed72f043e5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:42.770379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd87172-2aba-4acd-936f-bfed72f043e5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:42.843128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa149883-7f96-4f9d-9782-160e83756f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:42.843885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa149883-7f96-4f9d-9782-160e83756f10 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:42.844163 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa149883-7f96-4f9d-9782-160e83756f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:42.844464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa149883-7f96-4f9d-9782-160e83756f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:42.845357 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa149883-7f96-4f9d-9782-160e83756f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:42.847403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa149883-7f96-4f9d-9782-160e83756f10 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:42.865881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1020/2041] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:42.870687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12372497-ff5a-43ca-9f07-8183c7c9b65c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:42.882360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12372497-ff5a-43ca-9f07-8183c7c9b65c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:42.889631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:42.890819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:42.890986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:42.891273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] PATH_INFO: `/v3/projects/1e0ce8a87599448aba3c20e258c0eb50/users/29e612e6e6234fbda52532b6dbdba640/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:42.896118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0ce8a87599448aba3c20e258c0eb50, user_id=29e612e6e6234fbda52532b6dbdba640, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:42.899610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:42.913771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1022/2042] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:42 2022] PUT /identity/v3/projects/1e0ce8a87599448aba3c20e258c0eb50/users/29e612e6e6234fbda52532b6dbdba640/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:43.654525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30fc31-d352-4a1f-a16f-f052329c75b8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:43.655090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30fc31-d352-4a1f-a16f-f052329c75b8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:43.655090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30fc31-d352-4a1f-a16f-f052329c75b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:43.686477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6e30fc31-d352-4a1f-a16f-f052329c75b8 None None] MFA Rules not processed for user `29e612e6e6234fbda52532b6dbdba640`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:43.687714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6e30fc31-d352-4a1f-a16f-f052329c75b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:43.768946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1021/2043] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:44.081334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:44.083151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:44.153668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db11c1-7b93-4d0e-9c0a-518fcc5dc17d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:44.159537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:44.160523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.160810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.161088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] PATH_INFO: `/v3/users/ef82c6695c63440abbb336560efafa40` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.163068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef82c6695c63440abbb336560efafa40)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:44.164466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:44.217505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef82c6695c63440abbb336560efafa40'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.220971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:44.230900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef82c6695c63440abbb336560efafa40'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.250838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef82c6695c63440abbb336560efafa40'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.252431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef82c6695c63440abbb336560efafa40'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.253111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef82c6695c63440abbb336560efafa40'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:44.267289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1023/2044] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:44 2022] DELETE /identity/v3/users/ef82c6695c63440abbb336560efafa40 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:44.271933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06c48e-a287-4364-aff7-f4701a05abe3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.272239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06c48e-a287-4364-aff7-f4701a05abe3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.272526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06c48e-a287-4364-aff7-f4701a05abe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.291840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ff06c48e-a287-4364-aff7-f4701a05abe3 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:44.293232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ff06c48e-a287-4364-aff7-f4701a05abe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:44.350598 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:44.364974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1022/2045] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:44.372295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:44.374274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:44.400895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc13c-8435-4be9-a80e-32f95e88d23e None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.401352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc13c-8435-4be9-a80e-32f95e88d23e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.401846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcc13c-8435-4be9-a80e-32f95e88d23e None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.404482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1023/2046] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:20:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:20:44.414794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a74b-f400-4842-a518-a083184046e4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.415184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a74b-f400-4842-a518-a083184046e4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.415524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a74b-f400-4842-a518-a083184046e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.436103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d774a74b-f400-4842-a518-a083184046e4 None None] MFA Rules not processed for user `8895e6cf7a1e445c825f391309766d76`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:44.437472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d774a74b-f400-4842-a518-a083184046e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:44.472682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee5367-660c-427a-8cf5-a17e9a4d53bb admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:44.483504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:44.485194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.488119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.489065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.505175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:44.507027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:44.534010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1024/2047] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:44.550209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1024/2048] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:20:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:44.564734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:44.566673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:44.586655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59999e-b95a-44f1-b638-631b8d5fa73b service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:44.599349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:44.600983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:44.601630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:44.602049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:44.615953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:44.618669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:44.640489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1025/2049] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:20:45.184575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a74b-f400-4842-a518-a083184046e4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:45.186277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d774a74b-f400-4842-a518-a083184046e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:45.201398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a74b-f400-4842-a518-a083184046e4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:45.206758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:45.207945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.208314 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.214243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] PATH_INFO: `/v3/projects/0e7c99fbac26478a813421223e60bef9` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:45.217096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e7c99fbac26478a813421223e60bef9)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:45.218083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:45.280769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e7c99fbac26478a813421223e60bef9', 'request_id': 'req-928da213-01d9-4cf3-95bf-d2725beba699'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.290484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e7c99fbac26478a813421223e60bef9', 'request_id': 'req-928da213-01d9-4cf3-95bf-d2725beba699'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.302039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e7c99fbac26478a813421223e60bef9', 'request_id': 'req-928da213-01d9-4cf3-95bf-d2725beba699'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.305075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:45.307321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:45.325543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d5e1e-df46-4824-9a6c-103dfaa96891 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:45.329784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] The token cache is being invalidate because project 0e7c99fbac26478a813421223e60bef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:45.330098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.330776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.332780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1025/2050] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:45 2022] DELETE /identity/v3/projects/0e7c99fbac26478a813421223e60bef9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:45.336607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:45.338650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:45.406609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:45.407868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.408155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.408522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] PATH_INFO: `/v3/users/8dd275728e7a4a5b9ff524d122570c45` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:45.410054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd275728e7a4a5b9ff524d122570c45)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:45.412065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:45.432579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da213-01d9-4cf3-95bf-d2725beba699 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:45.438187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:45.439332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.439709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.440120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] PATH_INFO: `/v3/users/f886847b45aa4b94b82f510160c8ce50` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:45.441554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f886847b45aa4b94b82f510160c8ce50)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:45.443757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:45.462621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd275728e7a4a5b9ff524d122570c45'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.468571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:45.472974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd275728e7a4a5b9ff524d122570c45'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.479395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:45.491792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd275728e7a4a5b9ff524d122570c45'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.493043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd275728e7a4a5b9ff524d122570c45'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.493729 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd275728e7a4a5b9ff524d122570c45'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.509809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1026/2051] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:45 2022] DELETE /identity/v3/users/8dd275728e7a4a5b9ff524d122570c45 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:45.510934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f886847b45aa4b94b82f510160c8ce50'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.521084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f886847b45aa4b94b82f510160c8ce50'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.535242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f886847b45aa4b94b82f510160c8ce50'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.535991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f886847b45aa4b94b82f510160c8ce50'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.536611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f886847b45aa4b94b82f510160c8ce50'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.551693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1026/2052] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:45 2022] DELETE /identity/v3/users/f886847b45aa4b94b82f510160c8ce50 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:45.721607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:45.723328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:45.809483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e9e66-7d4c-4714-98f4-40d3e0887c83 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:45.894944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:45.895952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:45.896084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:45.896390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] PATH_INFO: `/v3/projects/dd6d368e19c740859b337513799b07e9` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:45.897602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6d368e19c740859b337513799b07e9)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:45.899737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:45.960224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6d368e19c740859b337513799b07e9', 'request_id': 'req-58f5d608-490a-4f3f-9d77-1432cbc6ea39'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.971322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:45.972821 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6d368e19c740859b337513799b07e9', 'request_id': 'req-58f5d608-490a-4f3f-9d77-1432cbc6ea39'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.973139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:45.981537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6d368e19c740859b337513799b07e9', 'request_id': 'req-58f5d608-490a-4f3f-9d77-1432cbc6ea39'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.994630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] The token cache is being invalidate because project dd6d368e19c740859b337513799b07e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:45.994958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.995338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094710f-9e8f-414f-9067-e2c5cc333f66 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:45.995587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:45.997445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1027/2053] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:45 2022] DELETE /identity/v3/projects/dd6d368e19c740859b337513799b07e9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:46.001349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.003442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.018147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:46.088931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.090090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.090495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.090820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] PATH_INFO: `/v3/projects/cc1aafa6c46d4a9abe69277c682e00d8` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.092195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1aafa6c46d4a9abe69277c682e00d8)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.094714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.095062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5d608-490a-4f3f-9d77-1432cbc6ea39 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.101878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.104214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.104629 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.105207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] PATH_INFO: `/v3/users/29e612e6e6234fbda52532b6dbdba640` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.106785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=29e612e6e6234fbda52532b6dbdba640)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.109017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.158930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1aafa6c46d4a9abe69277c682e00d8', 'request_id': 'req-eba12ee7-4cb5-420e-bb11-2b5067333417'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.167194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1aafa6c46d4a9abe69277c682e00d8', 'request_id': 'req-eba12ee7-4cb5-420e-bb11-2b5067333417'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.177985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1aafa6c46d4a9abe69277c682e00d8', 'request_id': 'req-eba12ee7-4cb5-420e-bb11-2b5067333417'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.186632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29e612e6e6234fbda52532b6dbdba640'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.192242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] The token cache is being invalidate because project cc1aafa6c46d4a9abe69277c682e00d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:46.193340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.194513 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-eba12ee7-4cb5-420e-bb11-2b5067333417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.196843 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29e612e6e6234fbda52532b6dbdba640'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.202408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1027/2054] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:45 2022] DELETE /identity/v3/projects/cc1aafa6c46d4a9abe69277c682e00d8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:46.204460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c852b38-6701-4bc7-8fd0-bbb4687a0f61 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.204946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c852b38-6701-4bc7-8fd0-bbb4687a0f61 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.205456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c852b38-6701-4bc7-8fd0-bbb4687a0f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.226748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29e612e6e6234fbda52532b6dbdba640'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.227502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29e612e6e6234fbda52532b6dbdba640'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.227640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7c852b38-6701-4bc7-8fd0-bbb4687a0f61 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:46.228262 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-71876d3d-09d2-47a2-97ba-06d8750cee9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29e612e6e6234fbda52532b6dbdba640'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.228818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7c852b38-6701-4bc7-8fd0-bbb4687a0f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.244097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1028/2055] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:46 2022] DELETE /identity/v3/users/29e612e6e6234fbda52532b6dbdba640 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:46.255796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:46.325500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1028/2056] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:46.335420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac320cb1-5855-447d-b984-a323e0c1e9ce None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.336025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac320cb1-5855-447d-b984-a323e0c1e9ce None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.336025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac320cb1-5855-447d-b984-a323e0c1e9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.356211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ac320cb1-5855-447d-b984-a323e0c1e9ce None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:46.357672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ac320cb1-5855-447d-b984-a323e0c1e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.464158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1029/2057] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:46.468759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c852b38-6701-4bc7-8fd0-bbb4687a0f61 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.481855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c852b38-6701-4bc7-8fd0-bbb4687a0f61 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.483269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac320cb1-5855-447d-b984-a323e0c1e9ce None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.484871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ac320cb1-5855-447d-b984-a323e0c1e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.487891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.489649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.489649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.489649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.490493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:46.490803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.492429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.504493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac320cb1-5855-447d-b984-a323e0c1e9ce None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.515308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.516262 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.516738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.517192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] PATH_INFO: `/v3/projects/1e0ce8a87599448aba3c20e258c0eb50` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.518565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e0ce8a87599448aba3c20e258c0eb50)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.520759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.526711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1029/2058] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:46.529646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.541816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0938-5cb6-4927-a603-d2c98b201e7c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.546853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59752116-e77a-4db0-8b6a-52b734214458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.547552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59752116-e77a-4db0-8b6a-52b734214458 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.547891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59752116-e77a-4db0-8b6a-52b734214458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.548210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59752116-e77a-4db0-8b6a-52b734214458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.548970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59752116-e77a-4db0-8b6a-52b734214458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.551009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59752116-e77a-4db0-8b6a-52b734214458 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.560336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:46.579496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1030/2059] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:46.581323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e0ce8a87599448aba3c20e258c0eb50', 'request_id': 'req-5746a146-58ef-4215-9a2e-1e83356afc3a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.583843 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59752116-e77a-4db0-8b6a-52b734214458 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.591594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e0ce8a87599448aba3c20e258c0eb50', 'request_id': 'req-5746a146-58ef-4215-9a2e-1e83356afc3a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.599446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e0ce8a87599448aba3c20e258c0eb50', 'request_id': 'req-5746a146-58ef-4215-9a2e-1e83356afc3a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.613239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] The token cache is being invalidate because project 1e0ce8a87599448aba3c20e258c0eb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:46.613937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.614643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5746a146-58ef-4215-9a2e-1e83356afc3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:46.619910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1030/2060] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:46 2022] DELETE /identity/v3/projects/1e0ce8a87599448aba3c20e258c0eb50 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:46.621610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e895d71-86ac-411b-b9f3-7b273d0fbdb3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.621945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e895d71-86ac-411b-b9f3-7b273d0fbdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.622217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e895d71-86ac-411b-b9f3-7b273d0fbdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.648744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6e895d71-86ac-411b-b9f3-7b273d0fbdb3 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:46.650053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6e895d71-86ac-411b-b9f3-7b273d0fbdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.662514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59752116-e77a-4db0-8b6a-52b734214458 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.667797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.668472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.668758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.669163 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.670049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.672355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.698080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1031/2061] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:46.701175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.702929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.715373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69699468-5867-4589-8e21-35fd304a35c8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.716338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1031/2062] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:46.720191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e9f3-0f01-49b4-90ea-42ae715b1883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.720860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e9f3-0f01-49b4-90ea-42ae715b1883 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.721140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e9f3-0f01-49b4-90ea-42ae715b1883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.721414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e9f3-0f01-49b4-90ea-42ae715b1883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.722186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e9f3-0f01-49b4-90ea-42ae715b1883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.724115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e9f3-0f01-49b4-90ea-42ae715b1883 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.725700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b1a42-33ae-448b-aac5-2156a47c53ac None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.726017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b1a42-33ae-448b-aac5-2156a47c53ac None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.726281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b1a42-33ae-448b-aac5-2156a47c53ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.739566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1032/2063] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:46.741419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e9f3-0f01-49b4-90ea-42ae715b1883 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.752305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e9f3-0f01-49b4-90ea-42ae715b1883 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.752984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ba7b1a42-33ae-448b-aac5-2156a47c53ac None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:46.754302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ba7b1a42-33ae-448b-aac5-2156a47c53ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.762255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1032/2064] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:46.766630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b1a42-33ae-448b-aac5-2156a47c53ac None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.772910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f64b6-749d-4dce-b735-0bde36fa8f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.773599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f64b6-749d-4dce-b735-0bde36fa8f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.773948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f64b6-749d-4dce-b735-0bde36fa8f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.774287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f64b6-749d-4dce-b735-0bde36fa8f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.775110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f64b6-749d-4dce-b735-0bde36fa8f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.776894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f64b6-749d-4dce-b735-0bde36fa8f8d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.777010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b1a42-33ae-448b-aac5-2156a47c53ac None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.783347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.784068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.784363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.784638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.785426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:46.785762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.787421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.791420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1033/2065] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:46.795446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f64b6-749d-4dce-b735-0bde36fa8f8d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.803659 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1033/2066] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:20:46.805218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.809695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f64b6-749d-4dce-b735-0bde36fa8f8d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.814474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d3e7e-894f-4f4f-9e6b-d320abe33c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.815766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d3e7e-894f-4f4f-9e6b-d320abe33c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.815894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414bcc49-ae00-4df0-9bab-832907dca66b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.815975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d3e7e-894f-4f4f-9e6b-d320abe33c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.815975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d3e7e-894f-4f4f-9e6b-d320abe33c69 admin admin] PATH_INFO: `/v3/projects/db3f28eb31dd4f08bf4631518bbe9f43/users/ccd275fbe2e944d4aa6a6b5d0c5d144d/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.820860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b89b74-b615-4971-b167-b9b2ef19d66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.821772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b89b74-b615-4971-b167-b9b2ef19d66e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.822090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b89b74-b615-4971-b167-b9b2ef19d66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.822416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b89b74-b615-4971-b167-b9b2ef19d66e admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.823147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b89b74-b615-4971-b167-b9b2ef19d66e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.825313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b89b74-b615-4971-b167-b9b2ef19d66e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.839283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d3e7e-894f-4f4f-9e6b-d320abe33c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3f28eb31dd4f08bf4631518bbe9f43, user_id=ccd275fbe2e944d4aa6a6b5d0c5d144d, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.842681 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d3e7e-894f-4f4f-9e6b-d320abe33c69 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.854232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1034/2067] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:46.858508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b89b74-b615-4971-b167-b9b2ef19d66e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.861987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1034/2068] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:46 2022] PUT /identity/v3/projects/db3f28eb31dd4f08bf4631518bbe9f43/users/ccd275fbe2e944d4aa6a6b5d0c5d144d/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:46.873198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009649b-f059-4c10-95c1-4771f0a4c89a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.874923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009649b-f059-4c10-95c1-4771f0a4c89a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.875551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009649b-f059-4c10-95c1-4771f0a4c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.904744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3009649b-f059-4c10-95c1-4771f0a4c89a None None] MFA Rules not processed for user `ccd275fbe2e944d4aa6a6b5d0c5d144d`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:46.905992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3009649b-f059-4c10-95c1-4771f0a4c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:46.931403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:46.935910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b89b74-b615-4971-b167-b9b2ef19d66e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.944803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce764f48-d6a0-43b0-9e39-571beff06607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:46.945509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce764f48-d6a0-43b0-9e39-571beff06607 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:46.945889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce764f48-d6a0-43b0-9e39-571beff06607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:46.946298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce764f48-d6a0-43b0-9e39-571beff06607 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:46.947251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce764f48-d6a0-43b0-9e39-571beff06607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:46.950192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce764f48-d6a0-43b0-9e39-571beff06607 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:46.980460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1035/2069] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/users => generated 278 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:46.984260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce764f48-d6a0-43b0-9e39-571beff06607 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:46.993694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:46.996727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce764f48-d6a0-43b0-9e39-571beff06607 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:46.999955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1035/2070] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:47.004014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f45a5a-60f9-4489-9596-5807d3ec5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.005599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009649b-f059-4c10-95c1-4771f0a4c89a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.005746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f45a5a-60f9-4489-9596-5807d3ec5514 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.005746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f45a5a-60f9-4489-9596-5807d3ec5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.006217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f45a5a-60f9-4489-9596-5807d3ec5514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.007185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f45a5a-60f9-4489-9596-5807d3ec5514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.009689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f45a5a-60f9-4489-9596-5807d3ec5514 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.018386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009649b-f059-4c10-95c1-4771f0a4c89a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.023730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1036/2071] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:47.023887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14b90f-40e7-4159-8aa7-587a5a8f2786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.024017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14b90f-40e7-4159-8aa7-587a5a8f2786 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.024256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14b90f-40e7-4159-8aa7-587a5a8f2786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.024542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14b90f-40e7-4159-8aa7-587a5a8f2786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.025048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f45a5a-60f9-4489-9596-5807d3ec5514 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.025387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14b90f-40e7-4159-8aa7-587a5a8f2786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.027479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14b90f-40e7-4159-8aa7-587a5a8f2786 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.035894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f45a5a-60f9-4489-9596-5807d3ec5514 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.040904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a6f3b-6b91-4650-be19-b4992208c9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.041742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a6f3b-6b91-4650-be19-b4992208c9da admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.042144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a6f3b-6b91-4650-be19-b4992208c9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.042520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a6f3b-6b91-4650-be19-b4992208c9da admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.043379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a6f3b-6b91-4650-be19-b4992208c9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.045182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a6f3b-6b91-4650-be19-b4992208c9da admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.058773 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1036/2072] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:47.064229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14b90f-40e7-4159-8aa7-587a5a8f2786 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.067125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1037/2073] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:47.068089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a6f3b-6b91-4650-be19-b4992208c9da admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.082866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a6f3b-6b91-4650-be19-b4992208c9da admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.148377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14b90f-40e7-4159-8aa7-587a5a8f2786 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.153216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bfcde-c7e4-43f3-a6ab-ef63748e2b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.154256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cc7ce-4ae7-4172-96d3-93ee47fb29be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.154441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bfcde-c7e4-43f3-a6ab-ef63748e2b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.154779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bfcde-c7e4-43f3-a6ab-ef63748e2b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.154952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cc7ce-4ae7-4172-96d3-93ee47fb29be admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.155089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bfcde-c7e4-43f3-a6ab-ef63748e2b19 admin admin] PATH_INFO: `/v3/projects/8b003d64804044f1854d1320c0839346/users/f0419a7978a540b69334cca0e7eb6e91/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.155239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cc7ce-4ae7-4172-96d3-93ee47fb29be admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.155494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1cc7ce-4ae7-4172-96d3-93ee47fb29be admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.156214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cc7ce-4ae7-4172-96d3-93ee47fb29be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.158733 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1cc7ce-4ae7-4172-96d3-93ee47fb29be admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.175396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bfcde-c7e4-43f3-a6ab-ef63748e2b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b003d64804044f1854d1320c0839346, user_id=f0419a7978a540b69334cca0e7eb6e91, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.178321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bfcde-c7e4-43f3-a6ab-ef63748e2b19 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.191361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1037/2074] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:47.196451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cc7ce-4ae7-4172-96d3-93ee47fb29be admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.204931 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1038/2075] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:47 2022] PUT /identity/v3/projects/8b003d64804044f1854d1320c0839346/users/f0419a7978a540b69334cca0e7eb6e91/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:47.210914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1cc7ce-4ae7-4172-96d3-93ee47fb29be admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.215386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01a80d-3f51-428a-a0d2-94d608f10d7f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.215686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01a80d-3f51-428a-a0d2-94d608f10d7f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.215958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01a80d-3f51-428a-a0d2-94d608f10d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.261257 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6c01a80d-3f51-428a-a0d2-94d608f10d7f None None] MFA Rules not processed for user `f0419a7978a540b69334cca0e7eb6e91`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:47.263169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c01a80d-3f51-428a-a0d2-94d608f10d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.302082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218de2-3c71-460d-bdbb-47c2cf686632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.303225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218de2-3c71-460d-bdbb-47c2cf686632 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.304060 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218de2-3c71-460d-bdbb-47c2cf686632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.304415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218de2-3c71-460d-bdbb-47c2cf686632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.305587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83218de2-3c71-460d-bdbb-47c2cf686632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.308095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83218de2-3c71-460d-bdbb-47c2cf686632 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.320955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1038/2076] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:47.325030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218de2-3c71-460d-bdbb-47c2cf686632 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.336647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218de2-3c71-460d-bdbb-47c2cf686632 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.341842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e388b48-d038-47cf-9664-38fef28e9059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.342921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e388b48-d038-47cf-9664-38fef28e9059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.343199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e388b48-d038-47cf-9664-38fef28e9059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.343482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e388b48-d038-47cf-9664-38fef28e9059 admin admin] PATH_INFO: `/v3/projects/488c1cdabb8b4aeda49e1951810b052e/users/598909addab547ec85f8b0db8f637e9a/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.349002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1039/2077] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:47.353749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01a80d-3f51-428a-a0d2-94d608f10d7f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.359986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e388b48-d038-47cf-9664-38fef28e9059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488c1cdabb8b4aeda49e1951810b052e, user_id=598909addab547ec85f8b0db8f637e9a, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.363941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e388b48-d038-47cf-9664-38fef28e9059 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.365510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01a80d-3f51-428a-a0d2-94d608f10d7f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.371424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb734807-43e0-4a93-9080-de7ae158bb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.372128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb734807-43e0-4a93-9080-de7ae158bb7d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.372475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb734807-43e0-4a93-9080-de7ae158bb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.372679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb734807-43e0-4a93-9080-de7ae158bb7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.373381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb734807-43e0-4a93-9080-de7ae158bb7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.375866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb734807-43e0-4a93-9080-de7ae158bb7d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.380913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1039/2078] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:47 2022] PUT /identity/v3/projects/488c1cdabb8b4aeda49e1951810b052e/users/598909addab547ec85f8b0db8f637e9a/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:47.385033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e388b48-d038-47cf-9664-38fef28e9059 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.393495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:47.407222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1040/2079] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/projects => generated 355 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:20:47.411690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb734807-43e0-4a93-9080-de7ae158bb7d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.462902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e388b48-d038-47cf-9664-38fef28e9059 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.469135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:47.510434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb734807-43e0-4a93-9080-de7ae158bb7d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.515458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbcbbd-da96-44d8-b435-ad64c5fdac2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.516151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbcbbd-da96-44d8-b435-ad64c5fdac2d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.516436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbcbbd-da96-44d8-b435-ad64c5fdac2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.516901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbcbbd-da96-44d8-b435-ad64c5fdac2d admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.517623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbcbbd-da96-44d8-b435-ad64c5fdac2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.519599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbcbbd-da96-44d8-b435-ad64c5fdac2d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.536328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937f02-4305-4d1b-92ff-01c351b4ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.537154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937f02-4305-4d1b-92ff-01c351b4ab8e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.537462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937f02-4305-4d1b-92ff-01c351b4ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.537777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937f02-4305-4d1b-92ff-01c351b4ab8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.538541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb937f02-4305-4d1b-92ff-01c351b4ab8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.540284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb937f02-4305-4d1b-92ff-01c351b4ab8e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.541248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1041/2080] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/users => generated 284 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:47.545294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbcbbd-da96-44d8-b435-ad64c5fdac2d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.554824 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1040/2081] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:47.558510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbcbbd-da96-44d8-b435-ad64c5fdac2d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.558811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937f02-4305-4d1b-92ff-01c351b4ab8e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.563485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812347ea-4d8b-466a-bccc-824b5dbd7571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.564444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812347ea-4d8b-466a-bccc-824b5dbd7571 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.564727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812347ea-4d8b-466a-bccc-824b5dbd7571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.565003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812347ea-4d8b-466a-bccc-824b5dbd7571 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.565778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812347ea-4d8b-466a-bccc-824b5dbd7571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.567466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812347ea-4d8b-466a-bccc-824b5dbd7571 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.569392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937f02-4305-4d1b-92ff-01c351b4ab8e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.574961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fc14a-0451-4100-aca0-1a0b2ff98e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.575997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fc14a-0451-4100-aca0-1a0b2ff98e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.576308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fc14a-0451-4100-aca0-1a0b2ff98e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.576583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fc14a-0451-4100-aca0-1a0b2ff98e1d admin admin] PATH_INFO: `/v3/projects/488c1cdabb8b4aeda49e1951810b052e/users/598909addab547ec85f8b0db8f637e9a/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.581946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1042/2082] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:47.583156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812347ea-4d8b-466a-bccc-824b5dbd7571 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.585110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fc14a-0451-4100-aca0-1a0b2ff98e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=488c1cdabb8b4aeda49e1951810b052e, user_id=598909addab547ec85f8b0db8f637e9a, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.588204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fc14a-0451-4100-aca0-1a0b2ff98e1d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.594124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812347ea-4d8b-466a-bccc-824b5dbd7571 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.599401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1041/2083] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:47 2022] PUT /identity/v3/projects/488c1cdabb8b4aeda49e1951810b052e/users/598909addab547ec85f8b0db8f637e9a/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:47.610648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a08fa-3d1d-4c02-88ab-84b7c92aa63f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.611161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a08fa-3d1d-4c02-88ab-84b7c92aa63f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.611566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a08fa-3d1d-4c02-88ab-84b7c92aa63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.640539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e16a08fa-3d1d-4c02-88ab-84b7c92aa63f None None] MFA Rules not processed for user `598909addab547ec85f8b0db8f637e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:47.641772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e16a08fa-3d1d-4c02-88ab-84b7c92aa63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.659401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c59e6-bacc-4935-81b8-f5d821a4f0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.660301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c59e6-bacc-4935-81b8-f5d821a4f0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.660576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c59e6-bacc-4935-81b8-f5d821a4f0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.660847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c59e6-bacc-4935-81b8-f5d821a4f0d2 admin admin] PATH_INFO: `/v3/projects/e840d917dee04fa7bbfece6b4e460a06/users/65a47cd6f2f04c92944e8b34cb59c4de/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.677251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c59e6-bacc-4935-81b8-f5d821a4f0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e840d917dee04fa7bbfece6b4e460a06, user_id=65a47cd6f2f04c92944e8b34cb59c4de, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.680200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c59e6-bacc-4935-81b8-f5d821a4f0d2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.700959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1043/2084] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:47 2022] PUT /identity/v3/projects/e840d917dee04fa7bbfece6b4e460a06/users/65a47cd6f2f04c92944e8b34cb59c4de/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:20:47.705207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c59e6-bacc-4935-81b8-f5d821a4f0d2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.718557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1042/2085] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:47.730094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a08fa-3d1d-4c02-88ab-84b7c92aa63f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.731540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e16a08fa-3d1d-4c02-88ab-84b7c92aa63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.754764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:47.777754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c59e6-bacc-4935-81b8-f5d821a4f0d2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.783481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb390a9-c4d4-412f-9152-c38645881994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.784181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb390a9-c4d4-412f-9152-c38645881994 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.784464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb390a9-c4d4-412f-9152-c38645881994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.784878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb390a9-c4d4-412f-9152-c38645881994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.785611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb390a9-c4d4-412f-9152-c38645881994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.787317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb390a9-c4d4-412f-9152-c38645881994 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.793375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a08fa-3d1d-4c02-88ab-84b7c92aa63f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.798825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1044/2086] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:20:47.802896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb390a9-c4d4-412f-9152-c38645881994 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.817680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb390a9-c4d4-412f-9152-c38645881994 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:47.821707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e13d2a-86de-4678-b797-9b613244aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.822679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13d2a-86de-4678-b797-9b613244aa6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.822957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13d2a-86de-4678-b797-9b613244aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.823235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13d2a-86de-4678-b797-9b613244aa6d admin admin] PATH_INFO: `/v3/projects/e840d917dee04fa7bbfece6b4e460a06/users/65a47cd6f2f04c92944e8b34cb59c4de/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.825336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e13d2a-86de-4678-b797-9b613244aa6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e840d917dee04fa7bbfece6b4e460a06, user_id=65a47cd6f2f04c92944e8b34cb59c4de, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.828162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e13d2a-86de-4678-b797-9b613244aa6d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.840436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1045/2087] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:47 2022] PUT /identity/v3/projects/e840d917dee04fa7bbfece6b4e460a06/users/65a47cd6f2f04c92944e8b34cb59c4de/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:47.853171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477b8b4-ddf3-41e6-937c-1eb621268dd0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.853563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477b8b4-ddf3-41e6-937c-1eb621268dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.853814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477b8b4-ddf3-41e6-937c-1eb621268dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.859423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:47.860098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:47.860521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:47.860967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:47.883150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7477b8b4-ddf3-41e6-937c-1eb621268dd0 None None] MFA Rules not processed for user `65a47cd6f2f04c92944e8b34cb59c4de`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:47.884681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7477b8b4-ddf3-41e6-937c-1eb621268dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.890450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:47.932147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:47.933643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:47.943918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1043/2088] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:20:47.962856 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1046/2089] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:47 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:47.975335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:47.976917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:47.989429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e99f1d-dc56-422d-8bd1-e81bedc483e3 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:48.054864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c7775-fce1-43ec-bea8-7468cff323e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:48.055544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7775-fce1-43ec-bea8-7468cff323e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.055834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7775-fce1-43ec-bea8-7468cff323e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.056362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7775-fce1-43ec-bea8-7468cff323e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.069674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c7775-fce1-43ec-bea8-7468cff323e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.071201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c7775-fce1-43ec-bea8-7468cff323e2 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.082516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1044/2090] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:47 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:48.171168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477b8b4-ddf3-41e6-937c-1eb621268dd0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:48.181596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477b8b4-ddf3-41e6-937c-1eb621268dd0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:48.185256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85944c4-3ada-4d80-b6a0-9ad381a438b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:48.187112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85944c4-3ada-4d80-b6a0-9ad381a438b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.187112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85944c4-3ada-4d80-b6a0-9ad381a438b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.187112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85944c4-3ada-4d80-b6a0-9ad381a438b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.260814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85944c4-3ada-4d80-b6a0-9ad381a438b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.262281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85944c4-3ada-4d80-b6a0-9ad381a438b6 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.275120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1047/2091] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:48 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:48.443899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c7775-fce1-43ec-bea8-7468cff323e2 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:48.484655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:48.527611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c7775-fce1-43ec-bea8-7468cff323e2 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:48.532671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:48.533762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.534102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.534378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] PATH_INFO: `/v3/users/ccd275fbe2e944d4aa6a6b5d0c5d144d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.535623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd275fbe2e944d4aa6a6b5d0c5d144d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.537703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.587894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd275fbe2e944d4aa6a6b5d0c5d144d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:48.596368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd275fbe2e944d4aa6a6b5d0c5d144d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:48.609479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd275fbe2e944d4aa6a6b5d0c5d144d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:48.610277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd275fbe2e944d4aa6a6b5d0c5d144d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:48.611545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd275fbe2e944d4aa6a6b5d0c5d144d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:48.627335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1045/2092] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:48 2022] DELETE /identity/v3/users/ccd275fbe2e944d4aa6a6b5d0c5d144d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:48.631931 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f490e2-38bb-429d-b616-610a5295f54a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.632395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f490e2-38bb-429d-b616-610a5295f54a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.632638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f490e2-38bb-429d-b616-610a5295f54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.655595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-74f490e2-38bb-429d-b616-610a5295f54a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:48.656826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-74f490e2-38bb-429d-b616-610a5295f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:48.725447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1048/2093] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:48.735454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:48.737125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:48.809360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d4c2d-447e-4c3f-b4b0-338fac27816e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:48.815527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bf2c6-a6f3-4586-9fe7-967b5c9a215a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:48.816188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bf2c6-a6f3-4586-9fe7-967b5c9a215a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:48.816473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bf2c6-a6f3-4586-9fe7-967b5c9a215a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:48.816749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279bf2c6-a6f3-4586-9fe7-967b5c9a215a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:48.832099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bf2c6-a6f3-4586-9fe7-967b5c9a215a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:48.833864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279bf2c6-a6f3-4586-9fe7-967b5c9a215a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:48.844557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1046/2094] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:49.194368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f490e2-38bb-429d-b616-610a5295f54a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:49.196162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-74f490e2-38bb-429d-b616-610a5295f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:49.211466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f490e2-38bb-429d-b616-610a5295f54a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:49.217562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:49.220727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:49.220727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:49.220727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] PATH_INFO: `/v3/projects/db3f28eb31dd4f08bf4631518bbe9f43` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:49.220727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3f28eb31dd4f08bf4631518bbe9f43)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:49.222945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:49.266042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:49.276839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3f28eb31dd4f08bf4631518bbe9f43', 'request_id': 'req-ea926673-e0bd-47a6-9d1a-c5f103c4475c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.286567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3f28eb31dd4f08bf4631518bbe9f43', 'request_id': 'req-ea926673-e0bd-47a6-9d1a-c5f103c4475c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.294142 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3f28eb31dd4f08bf4631518bbe9f43', 'request_id': 'req-ea926673-e0bd-47a6-9d1a-c5f103c4475c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.306951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] The token cache is being invalidate because project db3f28eb31dd4f08bf4631518bbe9f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:49.307317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.308054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.310228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1049/2095] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:49 2022] DELETE /identity/v3/projects/db3f28eb31dd4f08bf4631518bbe9f43 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:49.314388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bf2c6-a6f3-4586-9fe7-967b5c9a215a service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:49.324300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279bf2c6-a6f3-4586-9fe7-967b5c9a215a service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:49.379969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:49.392696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:49.393592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:49.393909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:49.394204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] PATH_INFO: `/v3/users/598909addab547ec85f8b0db8f637e9a` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:49.395465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=598909addab547ec85f8b0db8f637e9a)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:49.397443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:49.451758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '598909addab547ec85f8b0db8f637e9a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.466612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '598909addab547ec85f8b0db8f637e9a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.484468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '598909addab547ec85f8b0db8f637e9a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.485533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '598909addab547ec85f8b0db8f637e9a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.486707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-23946dd2-131d-4010-81db-60b6e386f092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '598909addab547ec85f8b0db8f637e9a'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:49.506890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1047/2096] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:49 2022] DELETE /identity/v3/users/598909addab547ec85f8b0db8f637e9a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:49.881576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:49.883283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:49.898368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea926673-e0bd-47a6-9d1a-c5f103c4475c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:49.961906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:49.962821 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:49.963110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:49.963390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] PATH_INFO: `/v3/projects/488c1cdabb8b4aeda49e1951810b052e` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:49.964645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=488c1cdabb8b4aeda49e1951810b052e)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:49.966631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.018400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '488c1cdabb8b4aeda49e1951810b052e', 'request_id': 'req-c220493b-455f-4bc0-9c3a-07b1a085f644'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:50.028305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '488c1cdabb8b4aeda49e1951810b052e', 'request_id': 'req-c220493b-455f-4bc0-9c3a-07b1a085f644'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:50.036320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '488c1cdabb8b4aeda49e1951810b052e', 'request_id': 'req-c220493b-455f-4bc0-9c3a-07b1a085f644'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:50.047342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] The token cache is being invalidate because project 488c1cdabb8b4aeda49e1951810b052e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:50.047634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:50.048474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c220493b-455f-4bc0-9c3a-07b1a085f644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:50.050327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1050/2097] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:49 2022] DELETE /identity/v3/projects/488c1cdabb8b4aeda49e1951810b052e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:50.056008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2213a47-7b62-4072-b263-f21a4e88fcf8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.056293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2213a47-7b62-4072-b263-f21a4e88fcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.056572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2213a47-7b62-4072-b263-f21a4e88fcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.076445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c2213a47-7b62-4072-b263-f21a4e88fcf8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:50.077691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c2213a47-7b62-4072-b263-f21a4e88fcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:50.156598 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1048/2098] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:50.227781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762c1e-5f53-47d0-9e97-fe5707119bff None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.228081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762c1e-5f53-47d0-9e97-fe5707119bff None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.228417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db762c1e-5f53-47d0-9e97-fe5707119bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.252294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-db762c1e-5f53-47d0-9e97-fe5707119bff None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:50.254011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-db762c1e-5f53-47d0-9e97-fe5707119bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:50.263108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1051/2099] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:50.267920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2213a47-7b62-4072-b263-f21a4e88fcf8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.279983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2213a47-7b62-4072-b263-f21a4e88fcf8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.285104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.285833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.286170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.286478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.287428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:20:50.287735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.289913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.305466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1049/2100] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:20:50.311196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762c1e-5f53-47d0-9e97-fe5707119bff None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.322598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db762c1e-5f53-47d0-9e97-fe5707119bff None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.328399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dd35e-04c3-41a2-a255-d485fbd0c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.329860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd35e-04c3-41a2-a255-d485fbd0c287 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.330545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd35e-04c3-41a2-a255-d485fbd0c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.330825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862dd35e-04c3-41a2-a255-d485fbd0c287 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.331542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dd35e-04c3-41a2-a255-d485fbd0c287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.333522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862dd35e-04c3-41a2-a255-d485fbd0c287 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.371733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1052/2101] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/projects => generated 361 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:50.376498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.387180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d49177-7abd-4923-a985-5b97939d653d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.447184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:50.454397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403283b-9e20-4670-a6eb-29cdcf9b068f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.455228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403283b-9e20-4670-a6eb-29cdcf9b068f admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.455992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403283b-9e20-4670-a6eb-29cdcf9b068f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.456204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403283b-9e20-4670-a6eb-29cdcf9b068f admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.456906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403283b-9e20-4670-a6eb-29cdcf9b068f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.459028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403283b-9e20-4670-a6eb-29cdcf9b068f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.486979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1050/2102] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/users => generated 281 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:20:50.491751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dd35e-04c3-41a2-a255-d485fbd0c287 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.502418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862dd35e-04c3-41a2-a255-d485fbd0c287 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.507154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820b2b6-1a32-40b6-a12d-2b20ab4ad701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.507862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820b2b6-1a32-40b6-a12d-2b20ab4ad701 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.508253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820b2b6-1a32-40b6-a12d-2b20ab4ad701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.508534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820b2b6-1a32-40b6-a12d-2b20ab4ad701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.509321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820b2b6-1a32-40b6-a12d-2b20ab4ad701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.511343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820b2b6-1a32-40b6-a12d-2b20ab4ad701 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.525118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1053/2103] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:50.529762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403283b-9e20-4670-a6eb-29cdcf9b068f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.540468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403283b-9e20-4670-a6eb-29cdcf9b068f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.546802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc27f6c-d70b-4734-a3bd-ea8161827280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.547447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc27f6c-d70b-4734-a3bd-ea8161827280 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.547943 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc27f6c-d70b-4734-a3bd-ea8161827280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.548215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc27f6c-d70b-4734-a3bd-ea8161827280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.548741 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc27f6c-d70b-4734-a3bd-ea8161827280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.550409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc27f6c-d70b-4734-a3bd-ea8161827280 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.561876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1051/2104] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:20:50.567636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820b2b6-1a32-40b6-a12d-2b20ab4ad701 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.578488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820b2b6-1a32-40b6-a12d-2b20ab4ad701 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.584355 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.585285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.585566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.585874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] PATH_INFO: `/v3/projects/e510631d355248189f53030fed762762/users/6b8c63bdbf4748fc9a550fa9d749ec74/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.598882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:50.602258 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e510631d355248189f53030fed762762, user_id=6b8c63bdbf4748fc9a550fa9d749ec74, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.605249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.621413 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1054/2105] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:20:50 2022] PUT /identity/v3/projects/e510631d355248189f53030fed762762/users/6b8c63bdbf4748fc9a550fa9d749ec74/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:50.632709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97c7c7-14c2-46c2-90fd-39f549766b7e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.633122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97c7c7-14c2-46c2-90fd-39f549766b7e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.633480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97c7c7-14c2-46c2-90fd-39f549766b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.668317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2f97c7c7-14c2-46c2-90fd-39f549766b7e None None] MFA Rules not processed for user `6b8c63bdbf4748fc9a550fa9d749ec74`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:50.669607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2f97c7c7-14c2-46c2-90fd-39f549766b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:50.750638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1052/2106] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:20:50.785064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.786818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:50.868261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97736a-2b23-4e38-a33f-95f0e8cf6409 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:50.872032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9f8c7-51a2-4b69-8cd6-05e4854e17d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:50.872697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9f8c7-51a2-4b69-8cd6-05e4854e17d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:50.872976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9f8c7-51a2-4b69-8cd6-05e4854e17d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:50.873256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9f8c7-51a2-4b69-8cd6-05e4854e17d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:50.884151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9f8c7-51a2-4b69-8cd6-05e4854e17d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:50.885823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9f8c7-51a2-4b69-8cd6-05e4854e17d1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:50.896046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1055/2107] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:50.917640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97c7c7-14c2-46c2-90fd-39f549766b7e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:50.996861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97c7c7-14c2-46c2-90fd-39f549766b7e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:51.006608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:51.007699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:51.007979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:51.008268 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] PATH_INFO: `/v3/users/6b8c63bdbf4748fc9a550fa9d749ec74` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:51.010457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b8c63bdbf4748fc9a550fa9d749ec74)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:51.012431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:51.071653 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b8c63bdbf4748fc9a550fa9d749ec74'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.081531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b8c63bdbf4748fc9a550fa9d749ec74'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.094447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:20:51.100899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b8c63bdbf4748fc9a550fa9d749ec74'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.102264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b8c63bdbf4748fc9a550fa9d749ec74'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.103150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b8c63bdbf4748fc9a550fa9d749ec74'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.120312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1053/2108] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:20:50 2022] DELETE /identity/v3/users/6b8c63bdbf4748fc9a550fa9d749ec74 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:20:51.124659 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3d182-29b0-4873-af82-0088d70ac423 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:51.125027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3d182-29b0-4873-af82-0088d70ac423 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:51.125371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3d182-29b0-4873-af82-0088d70ac423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:51.146956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0ec3d182-29b0-4873-af82-0088d70ac423 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:20:51.150354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0ec3d182-29b0-4873-af82-0088d70ac423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:51.230671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1056/2109] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:20:51.242102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:51.244085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:51.309937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27893736-b36e-41d1-867c-f88fd6ff68f3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:51.314136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:51.314708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:51.315000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:51.315278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:51.325441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:51.326926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:51.337507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1054/2110] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:20:51.651464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3d182-29b0-4873-af82-0088d70ac423 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:20:51.653615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0ec3d182-29b0-4873-af82-0088d70ac423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:20:51.668123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3d182-29b0-4873-af82-0088d70ac423 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:20:51.673896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:20:51.674784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:20:51.675090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:20:51.675398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] PATH_INFO: `/v3/projects/e510631d355248189f53030fed762762` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:20:51.677358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e510631d355248189f53030fed762762)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:20:51.679565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:20:51.737921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e510631d355248189f53030fed762762', 'request_id': 'req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.746944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e510631d355248189f53030fed762762', 'request_id': 'req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.754288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e510631d355248189f53030fed762762', 'request_id': 'req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.766746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] The token cache is being invalidate because project e510631d355248189f53030fed762762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:20:51.767064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.767807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2d03c40e-c7e6-4aba-8993-c8e747dcdf66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:20:51.769481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1057/2111] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:20:51 2022] DELETE /identity/v3/projects/e510631d355248189f53030fed762762 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:01.332575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=1,keystone:SELECT=32 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:01.762918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:03.692099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:03.693636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:03.707351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36e43d-433d-4907-9933-37ce15ff49af service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:03.770633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:03.771682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:03.772037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:03.772327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] PATH_INFO: `/v3/users/f0419a7978a540b69334cca0e7eb6e91` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:03.773642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0419a7978a540b69334cca0e7eb6e91)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:03.775720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:03.829600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0419a7978a540b69334cca0e7eb6e91'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:03.838772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0419a7978a540b69334cca0e7eb6e91'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:03.851748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0419a7978a540b69334cca0e7eb6e91'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:03.852445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0419a7978a540b69334cca0e7eb6e91'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:03.853052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0419a7978a540b69334cca0e7eb6e91'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:03.865470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1055/2112] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:03 2022] DELETE /identity/v3/users/f0419a7978a540b69334cca0e7eb6e91 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:03.871547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f12e2a-4dc9-4ed1-a52a-66542b0aa2e3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:03.872059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f12e2a-4dc9-4ed1-a52a-66542b0aa2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:03.872059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f12e2a-4dc9-4ed1-a52a-66542b0aa2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:03.898067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e5f12e2a-4dc9-4ed1-a52a-66542b0aa2e3 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:03.899298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e5f12e2a-4dc9-4ed1-a52a-66542b0aa2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:03.963946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1058/2113] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:03.977196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:03.979885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:04.048068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89b554-987c-4bd0-953a-dcb3f52d194a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:04.054384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5236c6-1770-44ec-9e5e-063665aed73d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:04.055054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5236c6-1770-44ec-9e5e-063665aed73d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:04.055337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5236c6-1770-44ec-9e5e-063665aed73d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:04.055609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5236c6-1770-44ec-9e5e-063665aed73d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:04.067249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5236c6-1770-44ec-9e5e-063665aed73d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:04.068699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5236c6-1770-44ec-9e5e-063665aed73d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:04.079475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1056/2114] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:04.364704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f12e2a-4dc9-4ed1-a52a-66542b0aa2e3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:04.367614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e5f12e2a-4dc9-4ed1-a52a-66542b0aa2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:04.383712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f12e2a-4dc9-4ed1-a52a-66542b0aa2e3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:04.390822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:04.391602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:04.391886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:04.392179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] PATH_INFO: `/v3/projects/8b003d64804044f1854d1320c0839346` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:04.393331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b003d64804044f1854d1320c0839346)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:04.395466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:04.450871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b003d64804044f1854d1320c0839346', 'request_id': 'req-3d026018-15a6-49d6-8884-774a757642f2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.460607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b003d64804044f1854d1320c0839346', 'request_id': 'req-3d026018-15a6-49d6-8884-774a757642f2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.467379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b003d64804044f1854d1320c0839346', 'request_id': 'req-3d026018-15a6-49d6-8884-774a757642f2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.480998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] The token cache is being invalidate because project 8b003d64804044f1854d1320c0839346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:04.481285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.482014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.483642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1059/2115] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:04 2022] DELETE /identity/v3/projects/8b003d64804044f1854d1320c0839346 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:04.487970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5236c6-1770-44ec-9e5e-063665aed73d service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:04.498355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5236c6-1770-44ec-9e5e-063665aed73d service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:04.517669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:04.566914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:04.567810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:04.568091 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:04.568377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] PATH_INFO: `/v3/users/65a47cd6f2f04c92944e8b34cb59c4de` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:04.569839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a47cd6f2f04c92944e8b34cb59c4de)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:04.571786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:04.621790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a47cd6f2f04c92944e8b34cb59c4de'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.630000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a47cd6f2f04c92944e8b34cb59c4de'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.643752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a47cd6f2f04c92944e8b34cb59c4de'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.644542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a47cd6f2f04c92944e8b34cb59c4de'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.645156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-f16e2f66-6680-4cbd-bddd-ffbd8c444c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a47cd6f2f04c92944e8b34cb59c4de'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:04.659882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1057/2116] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:04 2022] DELETE /identity/v3/users/65a47cd6f2f04c92944e8b34cb59c4de => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:04.982246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:04.984240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:05.004767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d026018-15a6-49d6-8884-774a757642f2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.073453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.074436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.074746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.075022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] PATH_INFO: `/v3/projects/e840d917dee04fa7bbfece6b4e460a06` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.076317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e840d917dee04fa7bbfece6b4e460a06)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.078353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.126996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:05.148106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e840d917dee04fa7bbfece6b4e460a06', 'request_id': 'req-37b2a561-ec95-445a-b172-047f434cd15e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.157106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e840d917dee04fa7bbfece6b4e460a06', 'request_id': 'req-37b2a561-ec95-445a-b172-047f434cd15e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.163779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e840d917dee04fa7bbfece6b4e460a06', 'request_id': 'req-37b2a561-ec95-445a-b172-047f434cd15e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.176149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] The token cache is being invalidate because project e840d917dee04fa7bbfece6b4e460a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:05.176440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.177270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-37b2a561-ec95-445a-b172-047f434cd15e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:05.179359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1060/2117] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:04 2022] DELETE /identity/v3/projects/e840d917dee04fa7bbfece6b4e460a06 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:21:05.186514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18b823-904d-4557-a778-49ed07ba25fd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.187033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18b823-904d-4557-a778-49ed07ba25fd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.187124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef18b823-904d-4557-a778-49ed07ba25fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.208780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ef18b823-904d-4557-a778-49ed07ba25fd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:05.210622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ef18b823-904d-4557-a778-49ed07ba25fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:05.277593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1058/2118] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:05.286007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93ac2-8ffa-422f-9eb7-4eef748364ef None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.286249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93ac2-8ffa-422f-9eb7-4eef748364ef None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.286531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93ac2-8ffa-422f-9eb7-4eef748364ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.307090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b3d93ac2-8ffa-422f-9eb7-4eef748364ef None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:05.308696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b3d93ac2-8ffa-422f-9eb7-4eef748364ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:05.318365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1061/2119] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:05.324867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18b823-904d-4557-a778-49ed07ba25fd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.335884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef18b823-904d-4557-a778-49ed07ba25fd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.342050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.342714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.342971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.343248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.344019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:21:05.344332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.345994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.359368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1059/2120] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:21:05.364905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93ac2-8ffa-422f-9eb7-4eef748364ef None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.375854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93ac2-8ffa-422f-9eb7-4eef748364ef None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.381930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daff4a9-2237-42bc-ba9f-c30a0b85e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.382807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daff4a9-2237-42bc-ba9f-c30a0b85e0dc admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.383094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daff4a9-2237-42bc-ba9f-c30a0b85e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.383367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daff4a9-2237-42bc-ba9f-c30a0b85e0dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.384075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daff4a9-2237-42bc-ba9f-c30a0b85e0dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.386043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daff4a9-2237-42bc-ba9f-c30a0b85e0dc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.422985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1062/2121] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:21:05 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:05.429108 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.440602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60c7cb-3675-4801-acd8-0e634f3e96d5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.468550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:05.503059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0a39d-bec5-4ee7-a9c2-344965bf30fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.503779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0a39d-bec5-4ee7-a9c2-344965bf30fa admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.504065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0a39d-bec5-4ee7-a9c2-344965bf30fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.504344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0a39d-bec5-4ee7-a9c2-344965bf30fa admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.505072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0a39d-bec5-4ee7-a9c2-344965bf30fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.507790 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0a39d-bec5-4ee7-a9c2-344965bf30fa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.531134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1060/2122] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:21:05 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:21:05.535603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daff4a9-2237-42bc-ba9f-c30a0b85e0dc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.547315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daff4a9-2237-42bc-ba9f-c30a0b85e0dc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.554607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15d34d-b5da-45e2-8857-d0a283dbaf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.555295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d34d-b5da-45e2-8857-d0a283dbaf2c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.555573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d34d-b5da-45e2-8857-d0a283dbaf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.555849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15d34d-b5da-45e2-8857-d0a283dbaf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.556583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15d34d-b5da-45e2-8857-d0a283dbaf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.558240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15d34d-b5da-45e2-8857-d0a283dbaf2c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.570167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1063/2123] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:05.575680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0a39d-bec5-4ee7-a9c2-344965bf30fa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.587393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0a39d-bec5-4ee7-a9c2-344965bf30fa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.593924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac60f1-b169-412f-a435-88be6939eae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.594605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac60f1-b169-412f-a435-88be6939eae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.594886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac60f1-b169-412f-a435-88be6939eae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.595158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac60f1-b169-412f-a435-88be6939eae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.595901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac60f1-b169-412f-a435-88be6939eae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.597508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac60f1-b169-412f-a435-88be6939eae1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.610804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1061/2124] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:05.615589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15d34d-b5da-45e2-8857-d0a283dbaf2c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.626691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15d34d-b5da-45e2-8857-d0a283dbaf2c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.639016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac550fbc-53a6-4d33-baa5-a3245480f889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.639432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac550fbc-53a6-4d33-baa5-a3245480f889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.639697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac550fbc-53a6-4d33-baa5-a3245480f889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.640020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac550fbc-53a6-4d33-baa5-a3245480f889 admin admin] PATH_INFO: `/v3/projects/5bbd4352bda04864b5ca749945a8f487/users/0e9cbdb2efd646edbf2d7c72e4b2e42c/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.658296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac550fbc-53a6-4d33-baa5-a3245480f889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbd4352bda04864b5ca749945a8f487, user_id=0e9cbdb2efd646edbf2d7c72e4b2e42c, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.660974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac550fbc-53a6-4d33-baa5-a3245480f889 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.676733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1064/2125] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:05 2022] PUT /identity/v3/projects/5bbd4352bda04864b5ca749945a8f487/users/0e9cbdb2efd646edbf2d7c72e4b2e42c/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:21:05.688709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa0c37-af0b-4dad-96f3-98ff19cfda60 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.689087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa0c37-af0b-4dad-96f3-98ff19cfda60 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.689511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa0c37-af0b-4dad-96f3-98ff19cfda60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.720423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9cfa0c37-af0b-4dad-96f3-98ff19cfda60 None None] MFA Rules not processed for user `0e9cbdb2efd646edbf2d7c72e4b2e42c`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:05.721627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9cfa0c37-af0b-4dad-96f3-98ff19cfda60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:05.791039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1062/2126] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:05.795952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac550fbc-53a6-4d33-baa5-a3245480f889 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.859922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac550fbc-53a6-4d33-baa5-a3245480f889 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.865461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88565ab1-f984-45a4-9bd9-7c6715f7a6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.866157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88565ab1-f984-45a4-9bd9-7c6715f7a6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.866442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88565ab1-f984-45a4-9bd9-7c6715f7a6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.866733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88565ab1-f984-45a4-9bd9-7c6715f7a6f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.867556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88565ab1-f984-45a4-9bd9-7c6715f7a6f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.869492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88565ab1-f984-45a4-9bd9-7c6715f7a6f2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:05.899147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1065/2127] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:21:05 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:05.904120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa0c37-af0b-4dad-96f3-98ff19cfda60 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:05.918818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa0c37-af0b-4dad-96f3-98ff19cfda60 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:05.980087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07861f-4a57-4e41-a9ac-9feb25f7eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:05.980680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07861f-4a57-4e41-a9ac-9feb25f7eb05 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:05.980965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07861f-4a57-4e41-a9ac-9feb25f7eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:05.981237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07861f-4a57-4e41-a9ac-9feb25f7eb05 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:05.981963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07861f-4a57-4e41-a9ac-9feb25f7eb05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:05.983914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07861f-4a57-4e41-a9ac-9feb25f7eb05 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:06.011726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1063/2128] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:21:05 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:06.016689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88565ab1-f984-45a4-9bd9-7c6715f7a6f2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:06.029891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88565ab1-f984-45a4-9bd9-7c6715f7a6f2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:06.035177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ab171-4f5e-4c2a-8b80-40d3f5ceb84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:06.035828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab171-4f5e-4c2a-8b80-40d3f5ceb84e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:06.036112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab171-4f5e-4c2a-8b80-40d3f5ceb84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:06.036385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ab171-4f5e-4c2a-8b80-40d3f5ceb84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:06.037115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ab171-4f5e-4c2a-8b80-40d3f5ceb84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:06.038983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38ab171-4f5e-4c2a-8b80-40d3f5ceb84e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:06.050706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1066/2129] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:06.056147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07861f-4a57-4e41-a9ac-9feb25f7eb05 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:06.068459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07861f-4a57-4e41-a9ac-9feb25f7eb05 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:06.074195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48543ac1-41d6-42d4-b780-09eeb6421eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:06.075174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48543ac1-41d6-42d4-b780-09eeb6421eee admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:06.075459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48543ac1-41d6-42d4-b780-09eeb6421eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:06.075792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48543ac1-41d6-42d4-b780-09eeb6421eee admin admin] PATH_INFO: `/v3/projects/cba61806088f46c8836b679818dbe480/users/07d23b8857054ceabb4d45853057ad6c/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:06.091769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48543ac1-41d6-42d4-b780-09eeb6421eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba61806088f46c8836b679818dbe480, user_id=07d23b8857054ceabb4d45853057ad6c, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:06.094513 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48543ac1-41d6-42d4-b780-09eeb6421eee admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:06.110456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1064/2130] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:06 2022] PUT /identity/v3/projects/cba61806088f46c8836b679818dbe480/users/07d23b8857054ceabb4d45853057ad6c/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:06.114817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ab171-4f5e-4c2a-8b80-40d3f5ceb84e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:06.127324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38ab171-4f5e-4c2a-8b80-40d3f5ceb84e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:06.149540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:06.191144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868960e-2cda-480b-8d16-dcf04b9bc047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:06.192111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868960e-2cda-480b-8d16-dcf04b9bc047 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:06.192401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868960e-2cda-480b-8d16-dcf04b9bc047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:06.192682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868960e-2cda-480b-8d16-dcf04b9bc047 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:06.193420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868960e-2cda-480b-8d16-dcf04b9bc047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:06.195348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868960e-2cda-480b-8d16-dcf04b9bc047 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:06.206165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1067/2131] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:06.210747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48543ac1-41d6-42d4-b780-09eeb6421eee admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:06.216791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:06.223043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48543ac1-41d6-42d4-b780-09eeb6421eee admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:06.229611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:06.230606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:06.230825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:06.231096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] PATH_INFO: `/v3/projects/cba61806088f46c8836b679818dbe480/users/07d23b8857054ceabb4d45853057ad6c/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:06.233068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba61806088f46c8836b679818dbe480, user_id=07d23b8857054ceabb4d45853057ad6c, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:06.235746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:06.247294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1065/2132] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:06 2022] PUT /identity/v3/projects/cba61806088f46c8836b679818dbe480/users/07d23b8857054ceabb4d45853057ad6c/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:06.263047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8514d31-55ec-4e20-ba3d-65716508cbff None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:06.263324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8514d31-55ec-4e20-ba3d-65716508cbff None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:06.263618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8514d31-55ec-4e20-ba3d-65716508cbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:06.291813 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a8514d31-55ec-4e20-ba3d-65716508cbff None None] MFA Rules not processed for user `07d23b8857054ceabb4d45853057ad6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:06.293102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a8514d31-55ec-4e20-ba3d-65716508cbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:06.373898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1068/2133] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:21:06.385939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:06.387904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:06.460882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fd3cb-1923-4c91-bf4f-209976aba367 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:06.464840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecac6c5-8522-46dc-b739-10251790bc45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:06.465548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecac6c5-8522-46dc-b739-10251790bc45 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:06.465853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecac6c5-8522-46dc-b739-10251790bc45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:06.466127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecac6c5-8522-46dc-b739-10251790bc45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:06.476076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecac6c5-8522-46dc-b739-10251790bc45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:06.477867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecac6c5-8522-46dc-b739-10251790bc45 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:06.488218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1066/2134] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:06.592279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8514d31-55ec-4e20-ba3d-65716508cbff None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:06.602371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8514d31-55ec-4e20-ba3d-65716508cbff None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:06.606148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508fab6-d983-4e87-b5b1-48f13731095b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:06.606823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508fab6-d983-4e87-b5b1-48f13731095b service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:06.607099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508fab6-d983-4e87-b5b1-48f13731095b service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:06.607363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508fab6-d983-4e87-b5b1-48f13731095b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:06.668212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508fab6-d983-4e87-b5b1-48f13731095b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:06.669771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508fab6-d983-4e87-b5b1-48f13731095b service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:06.679289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1069/2135] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:09.295378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecac6c5-8522-46dc-b739-10251790bc45 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:09.364026 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecac6c5-8522-46dc-b739-10251790bc45 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:09.369561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:09.370478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:09.370769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:09.371077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] PATH_INFO: `/v3/users/0e9cbdb2efd646edbf2d7c72e4b2e42c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:09.372361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e9cbdb2efd646edbf2d7c72e4b2e42c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:09.374465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:09.434503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e9cbdb2efd646edbf2d7c72e4b2e42c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.442964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e9cbdb2efd646edbf2d7c72e4b2e42c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.455811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e9cbdb2efd646edbf2d7c72e4b2e42c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.456481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e9cbdb2efd646edbf2d7c72e4b2e42c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.457095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e9cbdb2efd646edbf2d7c72e4b2e42c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:09.471436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1067/2136] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:09 2022] DELETE /identity/v3/users/0e9cbdb2efd646edbf2d7c72e4b2e42c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:09.476777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1471ef3-47fa-4106-8e7c-c2ffbbc110b9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:09.477339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1471ef3-47fa-4106-8e7c-c2ffbbc110b9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:09.477339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1471ef3-47fa-4106-8e7c-c2ffbbc110b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:09.496102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f1471ef3-47fa-4106-8e7c-c2ffbbc110b9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:09.497351 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f1471ef3-47fa-4106-8e7c-c2ffbbc110b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:09.541235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:09.561451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1070/2137] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:09.573696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:09.576000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:09.592462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:09.661537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65284ffa-fa0b-4968-a8fd-ede85a14b8b3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:09.668152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c8c41-94f4-43d3-907f-7ecf7e8e0d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:09.668805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c8c41-94f4-43d3-907f-7ecf7e8e0d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:09.669097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c8c41-94f4-43d3-907f-7ecf7e8e0d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:09.669384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c8c41-94f4-43d3-907f-7ecf7e8e0d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:09.681647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c8c41-94f4-43d3-907f-7ecf7e8e0d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:09.683052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c8c41-94f4-43d3-907f-7ecf7e8e0d8b service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:09.697217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1068/2138] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:10.302620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1471ef3-47fa-4106-8e7c-c2ffbbc110b9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:10.304626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f1471ef3-47fa-4106-8e7c-c2ffbbc110b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:10.318591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1471ef3-47fa-4106-8e7c-c2ffbbc110b9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:10.324407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:10.325237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:10.325518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:10.325834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] PATH_INFO: `/v3/projects/5bbd4352bda04864b5ca749945a8f487` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:10.327145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bbd4352bda04864b5ca749945a8f487)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:10.329097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:10.382816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bbd4352bda04864b5ca749945a8f487', 'request_id': 'req-1b735ea2-6973-46ea-963b-722cf450b143'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.391907 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bbd4352bda04864b5ca749945a8f487', 'request_id': 'req-1b735ea2-6973-46ea-963b-722cf450b143'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.398945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bbd4352bda04864b5ca749945a8f487', 'request_id': 'req-1b735ea2-6973-46ea-963b-722cf450b143'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.410884 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] The token cache is being invalidate because project 5bbd4352bda04864b5ca749945a8f487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:10.411176 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.411919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.413916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1071/2139] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:10 2022] DELETE /identity/v3/projects/5bbd4352bda04864b5ca749945a8f487 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:10.418243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c8c41-94f4-43d3-907f-7ecf7e8e0d8b service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:10.428826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c8c41-94f4-43d3-907f-7ecf7e8e0d8b service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:10.496396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:10.497284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:10.497561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:10.497869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] PATH_INFO: `/v3/users/07d23b8857054ceabb4d45853057ad6c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:10.499086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d23b8857054ceabb4d45853057ad6c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:10.501061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:10.554477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d23b8857054ceabb4d45853057ad6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.562277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d23b8857054ceabb4d45853057ad6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.576505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d23b8857054ceabb4d45853057ad6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.577129 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d23b8857054ceabb4d45853057ad6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.577778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e874f475-2d7d-43dc-8943-118357987cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d23b8857054ceabb4d45853057ad6c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:10.592926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1069/2140] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:10 2022] DELETE /identity/v3/users/07d23b8857054ceabb4d45853057ad6c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:10.917546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:10.919167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:10.933687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b735ea2-6973-46ea-963b-722cf450b143 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:10.994667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:10.995572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:10.995854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:10.996130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] PATH_INFO: `/v3/projects/cba61806088f46c8836b679818dbe480` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:10.997332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba61806088f46c8836b679818dbe480)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:10.999393 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.050906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba61806088f46c8836b679818dbe480', 'request_id': 'req-fca22b78-4411-41c4-885f-ba7191e38906'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:11.059598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba61806088f46c8836b679818dbe480', 'request_id': 'req-fca22b78-4411-41c4-885f-ba7191e38906'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:11.067012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba61806088f46c8836b679818dbe480', 'request_id': 'req-fca22b78-4411-41c4-885f-ba7191e38906'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:11.076529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:11.080683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] The token cache is being invalidate because project cba61806088f46c8836b679818dbe480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:11.081797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:11.082292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fca22b78-4411-41c4-885f-ba7191e38906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:11.084284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1072/2141] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:10 2022] DELETE /identity/v3/projects/cba61806088f46c8836b679818dbe480 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:21:11.089764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cb3c5-738c-4b62-a89f-9e6951d5990f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.090064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cb3c5-738c-4b62-a89f-9e6951d5990f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.090456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cb3c5-738c-4b62-a89f-9e6951d5990f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.111507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bd5cb3c5-738c-4b62-a89f-9e6951d5990f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:11.112769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bd5cb3c5-738c-4b62-a89f-9e6951d5990f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:11.146456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:11.184354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1070/2142] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:11.192477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19a693-38f1-473c-b445-6f37fca41870 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.192771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19a693-38f1-473c-b445-6f37fca41870 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.193052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19a693-38f1-473c-b445-6f37fca41870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.212031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6a19a693-38f1-473c-b445-6f37fca41870 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:11.213248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6a19a693-38f1-473c-b445-6f37fca41870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:11.222043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1073/2143] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:11.227247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cb3c5-738c-4b62-a89f-9e6951d5990f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.237287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cb3c5-738c-4b62-a89f-9e6951d5990f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.244329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.245040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.245327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.245637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.246399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:21:11.246713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.248359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.262758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1071/2144] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:21:11.267270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19a693-38f1-473c-b445-6f37fca41870 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.277949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19a693-38f1-473c-b445-6f37fca41870 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.283300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d16d9-d024-49f2-9c7c-5b5dc779af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.283978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d16d9-d024-49f2-9c7c-5b5dc779af79 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.284262 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d16d9-d024-49f2-9c7c-5b5dc779af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.284537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d16d9-d024-49f2-9c7c-5b5dc779af79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.285234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d16d9-d024-49f2-9c7c-5b5dc779af79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.287205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d16d9-d024-49f2-9c7c-5b5dc779af79 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.317643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1074/2145] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:21:11 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:11.322600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.336783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc35921-9797-476e-9bbb-6734ae851bc1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.400194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ac8a-b6b4-4d1c-a426-f01c14da729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.400872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ac8a-b6b4-4d1c-a426-f01c14da729c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.401158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ac8a-b6b4-4d1c-a426-f01c14da729c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.401431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0ac8a-b6b4-4d1c-a426-f01c14da729c admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.402173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0ac8a-b6b4-4d1c-a426-f01c14da729c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.404152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0ac8a-b6b4-4d1c-a426-f01c14da729c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.430684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1072/2146] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:21:11 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:11.435054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d16d9-d024-49f2-9c7c-5b5dc779af79 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.447371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d16d9-d024-49f2-9c7c-5b5dc779af79 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.454269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf10851-c4cb-4829-a596-a0f20b01e21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.455177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf10851-c4cb-4829-a596-a0f20b01e21e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.455465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf10851-c4cb-4829-a596-a0f20b01e21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.455737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf10851-c4cb-4829-a596-a0f20b01e21e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.456888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf10851-c4cb-4829-a596-a0f20b01e21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.458905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf10851-c4cb-4829-a596-a0f20b01e21e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.471950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1075/2147] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:11.476752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ac8a-b6b4-4d1c-a426-f01c14da729c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.491958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0ac8a-b6b4-4d1c-a426-f01c14da729c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.498021 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff607d-a725-4f6f-b8c8-f4198e54b1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.498714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff607d-a725-4f6f-b8c8-f4198e54b1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.498979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff607d-a725-4f6f-b8c8-f4198e54b1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.499261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff607d-a725-4f6f-b8c8-f4198e54b1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.500049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff607d-a725-4f6f-b8c8-f4198e54b1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.501686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff607d-a725-4f6f-b8c8-f4198e54b1bf admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.512951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1073/2148] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:11.517545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf10851-c4cb-4829-a596-a0f20b01e21e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.528454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf10851-c4cb-4829-a596-a0f20b01e21e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.533545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ab881-bda7-431c-95b5-0b98b95a6102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.534482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ab881-bda7-431c-95b5-0b98b95a6102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.534765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ab881-bda7-431c-95b5-0b98b95a6102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.535039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ab881-bda7-431c-95b5-0b98b95a6102 admin admin] PATH_INFO: `/v3/projects/fbc6d66723bc4b818d69554f7848c407/users/7537f82ef8954e58a7a7f84576429c88/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.551411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ab881-bda7-431c-95b5-0b98b95a6102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbc6d66723bc4b818d69554f7848c407, user_id=7537f82ef8954e58a7a7f84576429c88, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.554337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ab881-bda7-431c-95b5-0b98b95a6102 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.569264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1076/2149] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:11 2022] PUT /identity/v3/projects/fbc6d66723bc4b818d69554f7848c407/users/7537f82ef8954e58a7a7f84576429c88/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:11.583900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1aa32-b7ed-49d5-ba5b-308c61f666b8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.584259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1aa32-b7ed-49d5-ba5b-308c61f666b8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.584740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1aa32-b7ed-49d5-ba5b-308c61f666b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.612342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8ad1aa32-b7ed-49d5-ba5b-308c61f666b8 None None] MFA Rules not processed for user `7537f82ef8954e58a7a7f84576429c88`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:11.613921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8ad1aa32-b7ed-49d5-ba5b-308c61f666b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:11.688724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1074/2150] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:11.693324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ab881-bda7-431c-95b5-0b98b95a6102 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.761563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ab881-bda7-431c-95b5-0b98b95a6102 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.767652 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.768344 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.768606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.768885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.769588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.771606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.806511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1077/2151] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:21:11 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:11.812326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1aa32-b7ed-49d5-ba5b-308c61f666b8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.826926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1aa32-b7ed-49d5-ba5b-308c61f666b8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.856736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:11.895711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d7cc7-ec02-4310-94d0-982633645c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.896633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7cc7-ec02-4310-94d0-982633645c9c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.897014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7cc7-ec02-4310-94d0-982633645c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.897330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d7cc7-ec02-4310-94d0-982633645c9c admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.899190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7cc7-ec02-4310-94d0-982633645c9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.901717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d7cc7-ec02-4310-94d0-982633645c9c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.924219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1075/2152] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:21:11 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:11.927292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.929523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:11.930410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d7cc7-ec02-4310-94d0-982633645c9c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.941865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d7cc7-ec02-4310-94d0-982633645c9c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.942850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f32d6-a37e-4a6e-8400-847039671398 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.947799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.948242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a65a2-95fb-4334-8baa-f46347b4d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.948684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.948884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a65a2-95fb-4334-8baa-f46347b4d512 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.948999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.949223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] PATH_INFO: `/v3/users/73a0225f6d594018a19cec296937e2f8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.949452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a65a2-95fb-4334-8baa-f46347b4d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.949548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a65a2-95fb-4334-8baa-f46347b4d512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:11.950384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a65a2-95fb-4334-8baa-f46347b4d512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.950501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73a0225f6d594018a19cec296937e2f8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:11.952044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a65a2-95fb-4334-8baa-f46347b4d512 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.952338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:11.965246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1076/2153] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:11.970624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a65a2-95fb-4334-8baa-f46347b4d512 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:11.982205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a65a2-95fb-4334-8baa-f46347b4d512 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:11.988897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba752fa-a1e3-4611-acc4-845074944eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:11.990415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba752fa-a1e3-4611-acc4-845074944eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:11.990702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba752fa-a1e3-4611-acc4-845074944eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:11.991082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba752fa-a1e3-4611-acc4-845074944eb3 admin admin] PATH_INFO: `/v3/projects/82c6252429c4415e89f789b7902f62cf/users/725ae346dc324915997fcd163a768c6b/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.015566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba752fa-a1e3-4611-acc4-845074944eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c6252429c4415e89f789b7902f62cf, user_id=725ae346dc324915997fcd163a768c6b, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:12.018394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba752fa-a1e3-4611-acc4-845074944eb3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:12.025862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73a0225f6d594018a19cec296937e2f8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.035511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73a0225f6d594018a19cec296937e2f8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.036204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1077/2154] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:11 2022] PUT /identity/v3/projects/82c6252429c4415e89f789b7902f62cf/users/725ae346dc324915997fcd163a768c6b/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:21:12.040747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba752fa-a1e3-4611-acc4-845074944eb3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:12.049326 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73a0225f6d594018a19cec296937e2f8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.050077 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73a0225f6d594018a19cec296937e2f8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.050710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f7e2880-1998-4e1e-98d2-eee3bfc48ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73a0225f6d594018a19cec296937e2f8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.052797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:12.069705 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1078/2155] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:11 2022] DELETE /identity/v3/users/73a0225f6d594018a19cec296937e2f8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:12.074512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fd984-6d76-41c4-8756-a17592507bf4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.074862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fd984-6d76-41c4-8756-a17592507bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.075185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fd984-6d76-41c4-8756-a17592507bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.094575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ba7fd984-6d76-41c4-8756-a17592507bf4 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:12.095793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ba7fd984-6d76-41c4-8756-a17592507bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.108770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba752fa-a1e3-4611-acc4-845074944eb3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:12.158378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1079/2156] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:21:12.168350 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fd984-6d76-41c4-8756-a17592507bf4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:12.168676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:12.169473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.169836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ba7fd984-6d76-41c4-8756-a17592507bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.169948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.170068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.170797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:12.172447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:12.183878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1078/2157] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:21:12.187846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:12.189695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.202470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb5497-4eb9-4aeb-9eb1-68bd025a4329 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:12.207082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31c80c-c871-4540-83f8-ad2eadae0242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:12.207975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31c80c-c871-4540-83f8-ad2eadae0242 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.208257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31c80c-c871-4540-83f8-ad2eadae0242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.208530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31c80c-c871-4540-83f8-ad2eadae0242 admin admin] PATH_INFO: `/v3/projects/82c6252429c4415e89f789b7902f62cf/users/725ae346dc324915997fcd163a768c6b/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.210570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31c80c-c871-4540-83f8-ad2eadae0242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c6252429c4415e89f789b7902f62cf, user_id=725ae346dc324915997fcd163a768c6b, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:12.213258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31c80c-c871-4540-83f8-ad2eadae0242 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:12.224084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1079/2158] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:12 2022] PUT /identity/v3/projects/82c6252429c4415e89f789b7902f62cf/users/725ae346dc324915997fcd163a768c6b/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:12.234363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fd984-6d76-41c4-8756-a17592507bf4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:12.236203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad38aa-4301-4438-a52f-30628b5a56ef None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.236502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad38aa-4301-4438-a52f-30628b5a56ef None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.236779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad38aa-4301-4438-a52f-30628b5a56ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.263287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:12.268581 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3cad38aa-4301-4438-a52f-30628b5a56ef None None] MFA Rules not processed for user `725ae346dc324915997fcd163a768c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:12.270352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3cad38aa-4301-4438-a52f-30628b5a56ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.289882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:12.290962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.291251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.291527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.348355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1080/2159] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:12.359746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad38aa-4301-4438-a52f-30628b5a56ef None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:12.361307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3cad38aa-4301-4438-a52f-30628b5a56ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.370584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:12.372070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:12.380335 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:12.385998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1080/2160] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:21:12.432586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad38aa-4301-4438-a52f-30628b5a56ef None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:12.437149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e8c9c-c504-486a-a12c-098552969f29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:12.438065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e8c9c-c504-486a-a12c-098552969f29 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.438395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e8c9c-c504-486a-a12c-098552969f29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.439012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e8c9c-c504-486a-a12c-098552969f29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.440872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-414e8c9c-c504-486a-a12c-098552969f29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.514289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e8c9c-c504-486a-a12c-098552969f29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:12.515835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e8c9c-c504-486a-a12c-098552969f29 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:12.526874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1081/2161] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:21:12.719911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:12.721530 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.735707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90384de2-b4fe-4025-922a-674822b40228 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:12.752637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:12.753543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.753919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.754126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] PATH_INFO: `/v3/projects/fc310c704b1543be925d8764bac87320` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.755417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc310c704b1543be925d8764bac87320)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:12.758137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:12.809125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc310c704b1543be925d8764bac87320', 'request_id': 'req-8161a066-64dd-48d4-b2ec-e47cab5e3904'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.817841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc310c704b1543be925d8764bac87320', 'request_id': 'req-8161a066-64dd-48d4-b2ec-e47cab5e3904'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.825272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc310c704b1543be925d8764bac87320', 'request_id': 'req-8161a066-64dd-48d4-b2ec-e47cab5e3904'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.837168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] The token cache is being invalidate because project fc310c704b1543be925d8764bac87320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:12.837622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.838324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8161a066-64dd-48d4-b2ec-e47cab5e3904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:12.839957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1081/2162] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:12 2022] DELETE /identity/v3/projects/fc310c704b1543be925d8764bac87320 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:12.845149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de170d9-8e4d-4183-8d43-f842f683faa5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.845447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de170d9-8e4d-4183-8d43-f842f683faa5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.845762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de170d9-8e4d-4183-8d43-f842f683faa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.865460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7de170d9-8e4d-4183-8d43-f842f683faa5 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:12.867010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7de170d9-8e4d-4183-8d43-f842f683faa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.925980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:12.940151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1082/2163] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:12.948904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1323c03-f32e-4d08-9a67-91c0caecb8d1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:12.949477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1323c03-f32e-4d08-9a67-91c0caecb8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:12.949477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1323c03-f32e-4d08-9a67-91c0caecb8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:12.968956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a1323c03-f32e-4d08-9a67-91c0caecb8d1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:12.970129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a1323c03-f32e-4d08-9a67-91c0caecb8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:12.978953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1082/2164] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:12.983683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de170d9-8e4d-4183-8d43-f842f683faa5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:12.995360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de170d9-8e4d-4183-8d43-f842f683faa5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:13.003238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:13.003886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.004173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.004556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.005629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:21:13.006137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:13.008598 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:13.023766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1083/2165] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:21:13.028510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1323c03-f32e-4d08-9a67-91c0caecb8d1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:13.038669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1323c03-f32e-4d08-9a67-91c0caecb8d1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:13.043594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1fdd2-7934-4b7a-9db7-cd1b6317fd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:13.044371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1fdd2-7934-4b7a-9db7-cd1b6317fd27 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.044720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1fdd2-7934-4b7a-9db7-cd1b6317fd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.045066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1fdd2-7934-4b7a-9db7-cd1b6317fd27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.045971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1fdd2-7934-4b7a-9db7-cd1b6317fd27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:13.048047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1fdd2-7934-4b7a-9db7-cd1b6317fd27 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:13.076835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1083/2166] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:21:13 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:21:13.081503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:13.092744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fde91-716e-4b1f-9cb2-62117e9dbd9e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:13.154242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb6b72-9e4e-41dd-8582-d54f6643be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:13.154928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb6b72-9e4e-41dd-8582-d54f6643be59 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.155207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb6b72-9e4e-41dd-8582-d54f6643be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.155483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb6b72-9e4e-41dd-8582-d54f6643be59 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.156195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb6b72-9e4e-41dd-8582-d54f6643be59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:13.158197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb6b72-9e4e-41dd-8582-d54f6643be59 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:13.180203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1084/2167] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:21:13 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:13.185512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1fdd2-7934-4b7a-9db7-cd1b6317fd27 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:13.196643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1fdd2-7934-4b7a-9db7-cd1b6317fd27 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:13.202673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594e946-e6e7-442a-88a0-8a63ec21e98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:13.203569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594e946-e6e7-442a-88a0-8a63ec21e98d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.203889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594e946-e6e7-442a-88a0-8a63ec21e98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.204188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594e946-e6e7-442a-88a0-8a63ec21e98d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.205055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594e946-e6e7-442a-88a0-8a63ec21e98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:13.206841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594e946-e6e7-442a-88a0-8a63ec21e98d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:13.218949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1084/2168] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:13.223526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb6b72-9e4e-41dd-8582-d54f6643be59 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:13.234744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb6b72-9e4e-41dd-8582-d54f6643be59 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:13.240632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2346e-4129-44b1-b892-3e2d1a61dfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:13.241622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2346e-4129-44b1-b892-3e2d1a61dfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.242115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2346e-4129-44b1-b892-3e2d1a61dfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.242564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2346e-4129-44b1-b892-3e2d1a61dfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.243833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2346e-4129-44b1-b892-3e2d1a61dfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:13.245693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2346e-4129-44b1-b892-3e2d1a61dfb3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:13.257860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1085/2169] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:13.264240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594e946-e6e7-442a-88a0-8a63ec21e98d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:13.274550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594e946-e6e7-442a-88a0-8a63ec21e98d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:13.279284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:13.280186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.280460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.280732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] PATH_INFO: `/v3/projects/af54954fbeeb4772ad57290fceb3b177/users/a9057278ae1c42a5a2802a3be11828df/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.298677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af54954fbeeb4772ad57290fceb3b177, user_id=a9057278ae1c42a5a2802a3be11828df, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:13.300949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:13.315256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1085/2170] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:13 2022] PUT /identity/v3/projects/af54954fbeeb4772ad57290fceb3b177/users/a9057278ae1c42a5a2802a3be11828df/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:13.320525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25264b6f-15da-414c-b231-e90d0a712649 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.320819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25264b6f-15da-414c-b231-e90d0a712649 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.321096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25264b6f-15da-414c-b231-e90d0a712649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.343387 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-25264b6f-15da-414c-b231-e90d0a712649 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:13.345117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-25264b6f-15da-414c-b231-e90d0a712649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:13.412868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1086/2171] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:13.423877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:13.425764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:13.493050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c821e7-cfb8-4a9c-91d8-f7ba27ccb125 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:13.497470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdeea7-3ad5-4d4c-a2f4-c67fb91a6f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:13.498744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdeea7-3ad5-4d4c-a2f4-c67fb91a6f22 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:13.498744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdeea7-3ad5-4d4c-a2f4-c67fb91a6f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:13.498744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdeea7-3ad5-4d4c-a2f4-c67fb91a6f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:13.511372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdeea7-3ad5-4d4c-a2f4-c67fb91a6f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:13.511688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdeea7-3ad5-4d4c-a2f4-c67fb91a6f22 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:13.522048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1086/2172] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:17.075814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25264b6f-15da-414c-b231-e90d0a712649 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:17.079423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-25264b6f-15da-414c-b231-e90d0a712649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:17.175468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25264b6f-15da-414c-b231-e90d0a712649 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:17.183892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:17.184606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:17.184886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:17.185166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:17.188283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:17.214930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:17.299932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:17.302115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:17.320858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1087/2173] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:18.835306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb8f1-9b3c-4c79-9a60-36520de28244 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:18.835306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb8f1-9b3c-4c79-9a60-36520de28244 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:18.836240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cb8f1-9b3c-4c79-9a60-36520de28244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:18.877207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fc0cb8f1-9b3c-4c79-9a60-36520de28244 None None] MFA Rules not processed for user `a9057278ae1c42a5a2802a3be11828df`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:18.878697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fc0cb8f1-9b3c-4c79-9a60-36520de28244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:18.894210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:19.030151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1087/2174] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:19.035361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:19.055648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab5694-39fb-427b-a65d-499f667a669c service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:19.064631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e576657-fbcb-423d-91b5-a7344f417878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:19.065497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e576657-fbcb-423d-91b5-a7344f417878 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:19.066124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e576657-fbcb-423d-91b5-a7344f417878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:19.066474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e576657-fbcb-423d-91b5-a7344f417878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:19.067475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e576657-fbcb-423d-91b5-a7344f417878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:19.069688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e576657-fbcb-423d-91b5-a7344f417878 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:19.117508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1088/2175] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:21:19 2022] POST /identity/v3/projects => generated 383 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:21:19.123727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cb8f1-9b3c-4c79-9a60-36520de28244 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:19.138362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cb8f1-9b3c-4c79-9a60-36520de28244 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:19.355873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe557e-2ef6-4524-8560-3a2d8a83401d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:19.356751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe557e-2ef6-4524-8560-3a2d8a83401d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:19.357203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe557e-2ef6-4524-8560-3a2d8a83401d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:19.357929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe557e-2ef6-4524-8560-3a2d8a83401d admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:19.359152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe557e-2ef6-4524-8560-3a2d8a83401d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:19.361573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe557e-2ef6-4524-8560-3a2d8a83401d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:19.415977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1088/2176] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:21:19 2022] POST /identity/v3/users => generated 298 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:21:19.427698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e576657-fbcb-423d-91b5-a7344f417878 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:19.444186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e576657-fbcb-423d-91b5-a7344f417878 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:19.470988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56872e3-d585-43fe-b9a5-11820541229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:19.472717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56872e3-d585-43fe-b9a5-11820541229a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:19.473510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56872e3-d585-43fe-b9a5-11820541229a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:19.474099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56872e3-d585-43fe-b9a5-11820541229a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:19.475488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56872e3-d585-43fe-b9a5-11820541229a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:19.477853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56872e3-d585-43fe-b9a5-11820541229a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:19.493268 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1089/2177] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:19.502295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe557e-2ef6-4524-8560-3a2d8a83401d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:19.524132 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe557e-2ef6-4524-8560-3a2d8a83401d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:19.538170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395f928-0330-4521-8bdc-7c731789cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:19.538170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395f928-0330-4521-8bdc-7c731789cd36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:19.538520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395f928-0330-4521-8bdc-7c731789cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:19.539054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395f928-0330-4521-8bdc-7c731789cd36 admin admin] PATH_INFO: `/v3/projects/46ae3663cb1f4af7b69b88d1996d757f/users/18183f4a30774996baca9a555b84a007/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:19.601720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395f928-0330-4521-8bdc-7c731789cd36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ae3663cb1f4af7b69b88d1996d757f, user_id=18183f4a30774996baca9a555b84a007, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:19.609072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395f928-0330-4521-8bdc-7c731789cd36 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:19.662461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1089/2178] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:19 2022] PUT /identity/v3/projects/46ae3663cb1f4af7b69b88d1996d757f/users/18183f4a30774996baca9a555b84a007/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:19.671531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56872e3-d585-43fe-b9a5-11820541229a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:19.706627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56872e3-d585-43fe-b9a5-11820541229a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:19.843695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3faebe-72e4-45b2-a1ce-69427b250b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:19.843695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3faebe-72e4-45b2-a1ce-69427b250b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:19.843695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3faebe-72e4-45b2-a1ce-69427b250b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:19.843695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3faebe-72e4-45b2-a1ce-69427b250b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:19.844359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3faebe-72e4-45b2-a1ce-69427b250b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:19.846206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3faebe-72e4-45b2-a1ce-69427b250b9b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:19.861238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1090/2179] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:19.866293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395f928-0330-4521-8bdc-7c731789cd36 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:19.880315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395f928-0330-4521-8bdc-7c731789cd36 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:19.888549 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:19.889485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:19.889819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:19.890161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] PATH_INFO: `/v3/projects/46ae3663cb1f4af7b69b88d1996d757f/users/18183f4a30774996baca9a555b84a007/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:19.892639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46ae3663cb1f4af7b69b88d1996d757f, user_id=18183f4a30774996baca9a555b84a007, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:19.895557 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:19.925562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1090/2180] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:19 2022] PUT /identity/v3/projects/46ae3663cb1f4af7b69b88d1996d757f/users/18183f4a30774996baca9a555b84a007/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:25.852708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650a4f7-6368-434d-aa39-c784a2ef52e0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:25.853472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650a4f7-6368-434d-aa39-c784a2ef52e0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:25.853472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650a4f7-6368-434d-aa39-c784a2ef52e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:25.892348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e650a4f7-6368-434d-aa39-c784a2ef52e0 None None] MFA Rules not processed for user `18183f4a30774996baca9a555b84a007`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:25.893888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e650a4f7-6368-434d-aa39-c784a2ef52e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:25.986287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:25.998524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1091/2181] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:26.018048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:26.020628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:26.129412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c12f457-f070-4207-8066-1f2bba2b8367 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:26.138874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:26.139829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:26.140233 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:26.140567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:26.164090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:26.248888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:26.266990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:26.282823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1091/2182] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:26.494618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6c70-dc18-4cc8-b698-90ad00cd3a35 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:26.495034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6c70-dc18-4cc8-b698-90ad00cd3a35 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:26.495320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e6c70-dc18-4cc8-b698-90ad00cd3a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:26.516417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-602e6c70-dc18-4cc8-b698-90ad00cd3a35 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:26.517799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-602e6c70-dc18-4cc8-b698-90ad00cd3a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:26.581847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1092/2183] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:26.628634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:26.635816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:26.728655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a02ef-d8f5-4bc9-a326-c8bcd02135da service glance] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:26.736185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9bc4a-1ba9-42ff-830f-fc18c92786c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:26.736933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bc4a-1ba9-42ff-830f-fc18c92786c4 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:26.737574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bc4a-1ba9-42ff-830f-fc18c92786c4 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:26.737899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9bc4a-1ba9-42ff-830f-fc18c92786c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:26.753880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9bc4a-1ba9-42ff-830f-fc18c92786c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:26.755523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9bc4a-1ba9-42ff-830f-fc18c92786c4 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:26.777983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1092/2184] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:21:31.682175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e6c70-dc18-4cc8-b698-90ad00cd3a35 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:31.684815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-602e6c70-dc18-4cc8-b698-90ad00cd3a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:31.803265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e6c70-dc18-4cc8-b698-90ad00cd3a35 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:31.809447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:31.810883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:31.811394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:31.811874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] PATH_INFO: `/v3/users/7537f82ef8954e58a7a7f84576429c88` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:31.815457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7537f82ef8954e58a7a7f84576429c88)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:31.818176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:31.900277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7537f82ef8954e58a7a7f84576429c88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:31.923860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7537f82ef8954e58a7a7f84576429c88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:31.944854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7537f82ef8954e58a7a7f84576429c88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:31.945956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7537f82ef8954e58a7a7f84576429c88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:31.946914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7537f82ef8954e58a7a7f84576429c88'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:31.978175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1093/2185] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:31 2022] DELETE /identity/v3/users/7537f82ef8954e58a7a7f84576429c88 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:31.979653 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d258b-075c-4776-9e2e-2629096b6bc3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:31.980124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d258b-075c-4776-9e2e-2629096b6bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:31.980124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d258b-075c-4776-9e2e-2629096b6bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:32.001966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0a7d258b-075c-4776-9e2e-2629096b6bc3 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:32.003690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0a7d258b-075c-4776-9e2e-2629096b6bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:32.120791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1093/2186] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:32.130748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:32.132685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:32.171215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:32.270158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e65e5d-99ad-4122-b9c5-c125af53c1e6 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:32.284671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8076-250d-41fd-9776-845a52bf155c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:32.286499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8076-250d-41fd-9776-845a52bf155c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:32.286499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8076-250d-41fd-9776-845a52bf155c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:32.286728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8076-250d-41fd-9776-845a52bf155c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:32.300847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf8076-250d-41fd-9776-845a52bf155c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:32.303192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf8076-250d-41fd-9776-845a52bf155c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:32.338822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1094/2187] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:32.806116 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d258b-075c-4776-9e2e-2629096b6bc3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:32.807924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0a7d258b-075c-4776-9e2e-2629096b6bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:32.830596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d258b-075c-4776-9e2e-2629096b6bc3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:32.836286 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:32.837357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:32.837647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:32.838138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] PATH_INFO: `/v3/projects/fbc6d66723bc4b818d69554f7848c407` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:32.839572 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbc6d66723bc4b818d69554f7848c407)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:32.841502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:32.903259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:32.913919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbc6d66723bc4b818d69554f7848c407', 'request_id': 'req-8a4e78cf-7fd1-43b2-a0af-691711ede344'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:32.929599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbc6d66723bc4b818d69554f7848c407', 'request_id': 'req-8a4e78cf-7fd1-43b2-a0af-691711ede344'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:32.947183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbc6d66723bc4b818d69554f7848c407', 'request_id': 'req-8a4e78cf-7fd1-43b2-a0af-691711ede344'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:32.981004 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] The token cache is being invalidate because project fbc6d66723bc4b818d69554f7848c407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:32.981598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:32.982026 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:32.987166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1094/2188] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:32 2022] DELETE /identity/v3/projects/fbc6d66723bc4b818d69554f7848c407 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:32.991903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8076-250d-41fd-9776-845a52bf155c service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:33.004567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8076-250d-41fd-9776-845a52bf155c service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:33.104166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:33.105897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:33.106289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:33.106571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] PATH_INFO: `/v3/users/725ae346dc324915997fcd163a768c6b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:33.108103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=725ae346dc324915997fcd163a768c6b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:33.112120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:33.194765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725ae346dc324915997fcd163a768c6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:33.205531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725ae346dc324915997fcd163a768c6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:33.226576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725ae346dc324915997fcd163a768c6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:33.227400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725ae346dc324915997fcd163a768c6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:33.228079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-0c2aa6e6-092d-41b4-80db-2c00e5acf541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725ae346dc324915997fcd163a768c6b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:33.245685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1095/2189] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:32 2022] DELETE /identity/v3/users/725ae346dc324915997fcd163a768c6b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:33.626601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:33.628412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:33.647268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e78cf-7fd1-43b2-a0af-691711ede344 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:33.769172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:33.770436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:33.770966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:33.771617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] PATH_INFO: `/v3/projects/82c6252429c4415e89f789b7902f62cf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:33.775475 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c6252429c4415e89f789b7902f62cf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:33.778470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:33.894991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c6252429c4415e89f789b7902f62cf', 'request_id': 'req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:33.918945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c6252429c4415e89f789b7902f62cf', 'request_id': 'req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:33.937907 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c6252429c4415e89f789b7902f62cf', 'request_id': 'req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:33.985267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] The token cache is being invalidate because project 82c6252429c4415e89f789b7902f62cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:21:33.991973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:34.007965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ab86b0f2-e33e-4cfe-9cf8-16185acba4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:34.007965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1095/2190] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:21:33 2022] DELETE /identity/v3/projects/82c6252429c4415e89f789b7902f62cf => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:34.012906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cbfeb-9262-4a6e-a6d4-e5bd6e51042b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.014139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cbfeb-9262-4a6e-a6d4-e5bd6e51042b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.014139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cbfeb-9262-4a6e-a6d4-e5bd6e51042b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.039334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bc8cbfeb-9262-4a6e-a6d4-e5bd6e51042b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:34.040615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bc8cbfeb-9262-4a6e-a6d4-e5bd6e51042b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:34.092023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:34.117038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1096/2191] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:34.125592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df462e18-07d5-43c4-9ca5-b4bfc08d69be None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.125924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df462e18-07d5-43c4-9ca5-b4bfc08d69be None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.126212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df462e18-07d5-43c4-9ca5-b4bfc08d69be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.155709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-df462e18-07d5-43c4-9ca5-b4bfc08d69be None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:34.156981 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-df462e18-07d5-43c4-9ca5-b4bfc08d69be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:34.177758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1096/2192] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:34.181447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cbfeb-9262-4a6e-a6d4-e5bd6e51042b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:34.199726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cbfeb-9262-4a6e-a6d4-e5bd6e51042b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:34.207170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:34.207921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.208210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.208485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.210632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:21:34.210968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:34.212726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:34.229688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1097/2193] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:21:34.232078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df462e18-07d5-43c4-9ca5-b4bfc08d69be None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:34.248328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df462e18-07d5-43c4-9ca5-b4bfc08d69be None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:34.253303 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2295a53-a75e-4b71-88aa-991481732ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:34.254027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295a53-a75e-4b71-88aa-991481732ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.254311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295a53-a75e-4b71-88aa-991481732ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.254590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2295a53-a75e-4b71-88aa-991481732ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.255292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2295a53-a75e-4b71-88aa-991481732ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:34.258037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2295a53-a75e-4b71-88aa-991481732ac4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:34.289330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1097/2194] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:21:34 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:21:34.294047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:34.305119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b806489-d9f4-4c1a-bfdf-a571e3592578 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:34.420270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598a97d-43bb-4dcb-aef8-174b27b24b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:34.420858 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598a97d-43bb-4dcb-aef8-174b27b24b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.421177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598a97d-43bb-4dcb-aef8-174b27b24b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.421412 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598a97d-43bb-4dcb-aef8-174b27b24b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.422221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598a97d-43bb-4dcb-aef8-174b27b24b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:34.424316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598a97d-43bb-4dcb-aef8-174b27b24b1a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:34.470483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1098/2195] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:21:34 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:21:34.478276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2295a53-a75e-4b71-88aa-991481732ac4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:34.493178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2295a53-a75e-4b71-88aa-991481732ac4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:34.499436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef3ccf-2922-4112-a90e-a1b8b9c474fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:34.500495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3ccf-2922-4112-a90e-a1b8b9c474fc admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.500638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3ccf-2922-4112-a90e-a1b8b9c474fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.500918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef3ccf-2922-4112-a90e-a1b8b9c474fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.501733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef3ccf-2922-4112-a90e-a1b8b9c474fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:34.503471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef3ccf-2922-4112-a90e-a1b8b9c474fc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:34.516778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1098/2196] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:34.521610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598a97d-43bb-4dcb-aef8-174b27b24b1a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:34.538174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598a97d-43bb-4dcb-aef8-174b27b24b1a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:34.544878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd548fd-8049-47c3-be9a-5d8f94c753cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:34.545621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd548fd-8049-47c3-be9a-5d8f94c753cd admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.546112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd548fd-8049-47c3-be9a-5d8f94c753cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.546407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd548fd-8049-47c3-be9a-5d8f94c753cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.547213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd548fd-8049-47c3-be9a-5d8f94c753cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:34.548916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd548fd-8049-47c3-be9a-5d8f94c753cd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:34.561675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1099/2197] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:21:34.566139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef3ccf-2922-4112-a90e-a1b8b9c474fc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:34.578118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef3ccf-2922-4112-a90e-a1b8b9c474fc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:34.584451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:34.585405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.585746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.586033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] PATH_INFO: `/v3/projects/7e84b93e62424930b432e1d4eb97dd3f/users/4493a4f75e4043999acb28d46726796d/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.605894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e84b93e62424930b432e1d4eb97dd3f, user_id=4493a4f75e4043999acb28d46726796d, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:34.608732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:34.612955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:34.641488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1099/2198] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:21:34 2022] PUT /identity/v3/projects/7e84b93e62424930b432e1d4eb97dd3f/users/4493a4f75e4043999acb28d46726796d/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:21:34.653890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683ab5-fa47-472c-9c5e-f1744a5ce5e2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.654174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683ab5-fa47-472c-9c5e-f1744a5ce5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.654471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74683ab5-fa47-472c-9c5e-f1744a5ce5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.695465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-74683ab5-fa47-472c-9c5e-f1744a5ce5e2 None None] MFA Rules not processed for user `4493a4f75e4043999acb28d46726796d`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:34.697022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-74683ab5-fa47-472c-9c5e-f1744a5ce5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:34.792127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1100/2199] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:34.806130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:34.808228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:34.912244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835cd05-64ea-4f25-a32d-a8a031923eae admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:34.919345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523b3c9-4c1c-4c91-8abd-1ac645ab33b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:34.920338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523b3c9-4c1c-4c91-8abd-1ac645ab33b0 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:34.920625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523b3c9-4c1c-4c91-8abd-1ac645ab33b0 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:34.920906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523b3c9-4c1c-4c91-8abd-1ac645ab33b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:34.932370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523b3c9-4c1c-4c91-8abd-1ac645ab33b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:34.933880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523b3c9-4c1c-4c91-8abd-1ac645ab33b0 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:34.948404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1100/2200] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:21:38.515478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb2d69-7e44-4b38-930c-dd66d92a8b61 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:38.516496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb2d69-7e44-4b38-930c-dd66d92a8b61 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:38.516496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb2d69-7e44-4b38-930c-dd66d92a8b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:38.536835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-40eb2d69-7e44-4b38-930c-dd66d92a8b61 None None] MFA Rules not processed for user `a9057278ae1c42a5a2802a3be11828df`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:38.538309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-40eb2d69-7e44-4b38-930c-dd66d92a8b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:38.583496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:38.634435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1101/2201] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:41.298651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08217e8f-9a2a-4a69-9be3-e48b4286e48d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:41.299301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08217e8f-9a2a-4a69-9be3-e48b4286e48d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:41.299422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08217e8f-9a2a-4a69-9be3-e48b4286e48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:41.325128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-08217e8f-9a2a-4a69-9be3-e48b4286e48d None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:41.326565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-08217e8f-9a2a-4a69-9be3-e48b4286e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:41.415993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1101/2202] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:41.431141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb2d69-7e44-4b38-930c-dd66d92a8b61 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:41.433706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-40eb2d69-7e44-4b38-930c-dd66d92a8b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:41.562727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb2d69-7e44-4b38-930c-dd66d92a8b61 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:41.569383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e4837-80b8-4fd7-b068-b7b179eac7e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:41.570409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4837-80b8-4fd7-b068-b7b179eac7e9 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:41.570650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4837-80b8-4fd7-b068-b7b179eac7e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:41.570928 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e4837-80b8-4fd7-b068-b7b179eac7e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:41.580842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e4837-80b8-4fd7-b068-b7b179eac7e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:41.583592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e4837-80b8-4fd7-b068-b7b179eac7e9 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:41.596697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1102/2203] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:21:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:21:41.645470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a5511-9fac-4397-a646-43b9fcc2f9da None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:41.646635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a5511-9fac-4397-a646-43b9fcc2f9da None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:41.648437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a5511-9fac-4397-a646-43b9fcc2f9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:41.699229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d55a5511-9fac-4397-a646-43b9fcc2f9da None None] MFA Rules not processed for user `a0210f69010b4529822bcdac1bdf8b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:41.700478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d55a5511-9fac-4397-a646-43b9fcc2f9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:41.814030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1102/2204] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:51.591827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=39 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:51.796092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=78 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:21:57.863542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08aa54-d155-4310-9882-bbc8e94e3494 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:57.864197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08aa54-d155-4310-9882-bbc8e94e3494 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:57.864506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08aa54-d155-4310-9882-bbc8e94e3494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:57.893833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-dd08aa54-d155-4310-9882-bbc8e94e3494 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:57.898055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dd08aa54-d155-4310-9882-bbc8e94e3494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:57.913985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1103/2205] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:21:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:57.931056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a5511-9fac-4397-a646-43b9fcc2f9da None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:57.958590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a5511-9fac-4397-a646-43b9fcc2f9da None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:57.981137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d331a2-afe7-4e5d-b3a0-9b99b90669df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:57.982375 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d331a2-afe7-4e5d-b3a0-9b99b90669df service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:57.986062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d331a2-afe7-4e5d-b3a0-9b99b90669df service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:57.989012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d331a2-afe7-4e5d-b3a0-9b99b90669df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:58.021273 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d331a2-afe7-4e5d-b3a0-9b99b90669df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:58.023308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d331a2-afe7-4e5d-b3a0-9b99b90669df service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:58.055590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1103/2206] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:21:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:21:59.100324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08aa54-d155-4310-9882-bbc8e94e3494 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:59.286574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08aa54-d155-4310-9882-bbc8e94e3494 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:59.294610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:59.295944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:59.296431 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:59.296896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] PATH_INFO: `/v3/users/4493a4f75e4043999acb28d46726796d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:59.299506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4493a4f75e4043999acb28d46726796d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:21:59.302406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:21:59.491428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4493a4f75e4043999acb28d46726796d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:59.505218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4493a4f75e4043999acb28d46726796d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:59.534413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4493a4f75e4043999acb28d46726796d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:59.535197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4493a4f75e4043999acb28d46726796d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:59.535825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4493a4f75e4043999acb28d46726796d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:21:59.572599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1104/2207] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:21:59 2022] DELETE /identity/v3/users/4493a4f75e4043999acb28d46726796d => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:21:59.575838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2637672-4795-4948-9b95-5c71576bb6b5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:59.576483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2637672-4795-4948-9b95-5c71576bb6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:59.576825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2637672-4795-4948-9b95-5c71576bb6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:21:59.608043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b2637672-4795-4948-9b95-5c71576bb6b5 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:21:59.610214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b2637672-4795-4948-9b95-5c71576bb6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:59.817621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1104/2208] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:21:59.823253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:21:59.824770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:21:59.971705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8723ae12-e217-41bf-8dfe-5cc61101cad7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:21:59.976859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ddd82-8fe8-4383-b810-730b8cda104e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:21:59.977912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ddd82-8fe8-4383-b810-730b8cda104e service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:21:59.978227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ddd82-8fe8-4383-b810-730b8cda104e service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:21:59.978504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ddd82-8fe8-4383-b810-730b8cda104e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:00.009793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ddd82-8fe8-4383-b810-730b8cda104e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:00.009793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ddd82-8fe8-4383-b810-730b8cda104e service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:00.016500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1105/2209] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:00.649614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2637672-4795-4948-9b95-5c71576bb6b5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:00.649614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b2637672-4795-4948-9b95-5c71576bb6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:00.701728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2637672-4795-4948-9b95-5c71576bb6b5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:00.745463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:00.746727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:00.747295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:00.747757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] PATH_INFO: `/v3/projects/7e84b93e62424930b432e1d4eb97dd3f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:00.750354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e84b93e62424930b432e1d4eb97dd3f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:00.752761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:00.869253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e84b93e62424930b432e1d4eb97dd3f', 'request_id': 'req-791ee272-7355-4846-9726-ff7184836de2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:00.889985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e84b93e62424930b432e1d4eb97dd3f', 'request_id': 'req-791ee272-7355-4846-9726-ff7184836de2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:00.902341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e84b93e62424930b432e1d4eb97dd3f', 'request_id': 'req-791ee272-7355-4846-9726-ff7184836de2'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:00.926437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] The token cache is being invalidate because project 7e84b93e62424930b432e1d4eb97dd3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:22:00.926934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:00.927973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-791ee272-7355-4846-9726-ff7184836de2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:00.931714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1105/2210] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:22:00 2022] DELETE /identity/v3/projects/7e84b93e62424930b432e1d4eb97dd3f => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:22:00.936947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db9764-5d85-49ae-9a71-0401842f3d88 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:00.937899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db9764-5d85-49ae-9a71-0401842f3d88 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:00.938364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db9764-5d85-49ae-9a71-0401842f3d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:00.983601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d3db9764-5d85-49ae-9a71-0401842f3d88 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:00.984907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d3db9764-5d85-49ae-9a71-0401842f3d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:01.015026 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:01.158647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1106/2211] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:01.167554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1ccc9-295d-4cea-b24b-31d21599ed72 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:01.167887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1ccc9-295d-4cea-b24b-31d21599ed72 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:01.168137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1ccc9-295d-4cea-b24b-31d21599ed72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:01.205154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-61c1ccc9-295d-4cea-b24b-31d21599ed72 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:01.207278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-61c1ccc9-295d-4cea-b24b-31d21599ed72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:01.222007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1106/2212] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:01.229281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db9764-5d85-49ae-9a71-0401842f3d88 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:01.250638 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db9764-5d85-49ae-9a71-0401842f3d88 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:01.256844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:01.258586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:01.259089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:01.259524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:01.260520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:22:01.261000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:01.262887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:01.283264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1107/2213] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:22:01.289104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1ccc9-295d-4cea-b24b-31d21599ed72 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:01.317496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1ccc9-295d-4cea-b24b-31d21599ed72 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:01.341472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc31e1-5cdd-4d79-99f5-6f8e41262df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:01.342211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc31e1-5cdd-4d79-99f5-6f8e41262df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:01.345743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc31e1-5cdd-4d79-99f5-6f8e41262df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:01.345743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc31e1-5cdd-4d79-99f5-6f8e41262df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:01.345743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc31e1-5cdd-4d79-99f5-6f8e41262df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:01.358225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc31e1-5cdd-4d79-99f5-6f8e41262df9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:01.434227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1107/2214] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:22:01 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:22:01.438995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:01.468951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d71af-9b14-47e7-bf04-16b13be89b66 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:01.635635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff88b3-ba88-42c2-aafd-2b260cf5722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:01.637881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff88b3-ba88-42c2-aafd-2b260cf5722f admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:01.638286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff88b3-ba88-42c2-aafd-2b260cf5722f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:01.638665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff88b3-ba88-42c2-aafd-2b260cf5722f admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:01.640435 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff88b3-ba88-42c2-aafd-2b260cf5722f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:01.642819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff88b3-ba88-42c2-aafd-2b260cf5722f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:01.698820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1108/2215] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:22:01 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:22:01.703641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc31e1-5cdd-4d79-99f5-6f8e41262df9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:01.732117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc31e1-5cdd-4d79-99f5-6f8e41262df9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:01.741871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb04c5e-82fe-49aa-968c-8d63c9af39a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:01.743885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb04c5e-82fe-49aa-968c-8d63c9af39a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:01.744725 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb04c5e-82fe-49aa-968c-8d63c9af39a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:01.745329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb04c5e-82fe-49aa-968c-8d63c9af39a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:01.747221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb04c5e-82fe-49aa-968c-8d63c9af39a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:01.750847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb04c5e-82fe-49aa-968c-8d63c9af39a9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:01.779937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1108/2216] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:22:01.784663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff88b3-ba88-42c2-aafd-2b260cf5722f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:01.806257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff88b3-ba88-42c2-aafd-2b260cf5722f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:01.814807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaeee30-5bea-4b76-a7a4-6644139c1fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:01.816647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeee30-5bea-4b76-a7a4-6644139c1fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:01.817541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeee30-5bea-4b76-a7a4-6644139c1fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:01.818498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaeee30-5bea-4b76-a7a4-6644139c1fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:01.823239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaeee30-5bea-4b76-a7a4-6644139c1fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:01.826703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaeee30-5bea-4b76-a7a4-6644139c1fd6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:01.851729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1109/2217] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:22:01.853456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb04c5e-82fe-49aa-968c-8d63c9af39a9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:01.870230 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb04c5e-82fe-49aa-968c-8d63c9af39a9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:01.879865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:01.882448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:01.883155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:01.883568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] PATH_INFO: `/v3/projects/368eaa5764d042e49f04657cfd305709/users/7e6a114ca915457081f3a0459a8b6d69/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:01.909599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] RBAC: Authorizing `identity:create_grant(project_id=368eaa5764d042e49f04657cfd305709, user_id=7e6a114ca915457081f3a0459a8b6d69, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:01.919071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:01.961241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1109/2218] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:22:01 2022] PUT /identity/v3/projects/368eaa5764d042e49f04657cfd305709/users/7e6a114ca915457081f3a0459a8b6d69/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:22:01.986636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19387b8c-053e-442b-a183-0730bcea7939 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:01.988204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19387b8c-053e-442b-a183-0730bcea7939 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:01.988648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19387b8c-053e-442b-a183-0730bcea7939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:02.145783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-19387b8c-053e-442b-a183-0730bcea7939 None None] MFA Rules not processed for user `7e6a114ca915457081f3a0459a8b6d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:02.147876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-19387b8c-053e-442b-a183-0730bcea7939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:02.347691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1110/2219] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:02.363149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:02.366585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:02.389465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:02.536346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db291da8-d038-46c0-a6d7-daef9f53853b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:02.542213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c737abb-85f8-40bb-9048-5f01a9893cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:02.542950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c737abb-85f8-40bb-9048-5f01a9893cda service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:02.543418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c737abb-85f8-40bb-9048-5f01a9893cda service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:02.543790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c737abb-85f8-40bb-9048-5f01a9893cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:02.583465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c737abb-85f8-40bb-9048-5f01a9893cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:02.586404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c737abb-85f8-40bb-9048-5f01a9893cda service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:02.601397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1110/2220] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:12.342819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:12.594041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=23 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:15.438734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19387b8c-053e-442b-a183-0730bcea7939 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:15.523629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19387b8c-053e-442b-a183-0730bcea7939 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:15.528789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:15.529698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:15.529989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:15.530286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] PATH_INFO: `/v3/users/7e6a114ca915457081f3a0459a8b6d69` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:15.531668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6a114ca915457081f3a0459a8b6d69)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:15.533680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:15.588419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6a114ca915457081f3a0459a8b6d69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:15.599227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6a114ca915457081f3a0459a8b6d69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:15.613397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6a114ca915457081f3a0459a8b6d69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:15.614227 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6a114ca915457081f3a0459a8b6d69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:15.616186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6a114ca915457081f3a0459a8b6d69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:15.633355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1111/2221] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:22:15 2022] DELETE /identity/v3/users/7e6a114ca915457081f3a0459a8b6d69 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:22:15.641603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec876e80-da97-44d7-8601-2a9fa647d6e0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:15.642177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec876e80-da97-44d7-8601-2a9fa647d6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:15.642177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec876e80-da97-44d7-8601-2a9fa647d6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:15.663282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ec876e80-da97-44d7-8601-2a9fa647d6e0 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:15.669806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ec876e80-da97-44d7-8601-2a9fa647d6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:15.742277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1111/2222] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:15.749167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:15.750938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:15.811064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa81d-64bd-44c2-a8ec-1ea094d5a5fd admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:15.815027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0abc3-9693-4a8d-b247-e8166ebde391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:15.815693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0abc3-9693-4a8d-b247-e8166ebde391 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:15.815975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0abc3-9693-4a8d-b247-e8166ebde391 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:15.816252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0abc3-9693-4a8d-b247-e8166ebde391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:15.831454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0abc3-9693-4a8d-b247-e8166ebde391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:15.833013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0abc3-9693-4a8d-b247-e8166ebde391 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:15.844943 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1112/2223] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:16.238324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec876e80-da97-44d7-8601-2a9fa647d6e0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:16.240578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ec876e80-da97-44d7-8601-2a9fa647d6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:16.253620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec876e80-da97-44d7-8601-2a9fa647d6e0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:16.259248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:16.260257 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.260540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.260812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] PATH_INFO: `/v3/projects/368eaa5764d042e49f04657cfd305709` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.262064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=368eaa5764d042e49f04657cfd305709)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:16.274871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:16.364803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '368eaa5764d042e49f04657cfd305709', 'request_id': 'req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:16.377235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '368eaa5764d042e49f04657cfd305709', 'request_id': 'req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:16.393465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '368eaa5764d042e49f04657cfd305709', 'request_id': 'req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:16.407027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] The token cache is being invalidate because project 368eaa5764d042e49f04657cfd305709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:22:16.407395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:16.408129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-89fa7504-0b0d-485e-aa1b-5bb7a7000a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:16.410529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1112/2224] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:22:16 2022] DELETE /identity/v3/projects/368eaa5764d042e49f04657cfd305709 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:22:16.415814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96defc-b14d-4d1b-835d-159c69019340 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.416106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96defc-b14d-4d1b-835d-159c69019340 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.416410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96defc-b14d-4d1b-835d-159c69019340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.441748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5f96defc-b14d-4d1b-835d-159c69019340 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:16.443826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5f96defc-b14d-4d1b-835d-159c69019340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:16.462449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:16.513873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1113/2225] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:16.522317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d28358-ac82-4272-a624-3dbd05e7db9e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.522607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d28358-ac82-4272-a624-3dbd05e7db9e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.522891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d28358-ac82-4272-a624-3dbd05e7db9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.542429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f9d28358-ac82-4272-a624-3dbd05e7db9e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:16.543658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f9d28358-ac82-4272-a624-3dbd05e7db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:16.553264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1113/2226] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:16.557815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96defc-b14d-4d1b-835d-159c69019340 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:16.569528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96defc-b14d-4d1b-835d-159c69019340 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:16.574579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:16.575249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.575533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.575954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.576673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:22:16.577013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:16.578704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:16.593404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1114/2227] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:22:16.597969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d28358-ac82-4272-a624-3dbd05e7db9e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:16.608412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d28358-ac82-4272-a624-3dbd05e7db9e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:16.615753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c81e6-1131-4c09-b9a1-809f6c3bd84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:16.616610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c81e6-1131-4c09-b9a1-809f6c3bd84a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.616939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c81e6-1131-4c09-b9a1-809f6c3bd84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.617245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3c81e6-1131-4c09-b9a1-809f6c3bd84a admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.618419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c81e6-1131-4c09-b9a1-809f6c3bd84a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:16.621224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3c81e6-1131-4c09-b9a1-809f6c3bd84a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:16.659960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1114/2228] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:22:16 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:22:16.661259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:16.673064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1526e-f8ba-4ed8-8008-7af0f036c3d9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:16.738986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857df2bf-8e21-487c-be0a-d84f86636e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:16.739786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df2bf-8e21-487c-be0a-d84f86636e95 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.740192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df2bf-8e21-487c-be0a-d84f86636e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.740413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857df2bf-8e21-487c-be0a-d84f86636e95 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.741124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857df2bf-8e21-487c-be0a-d84f86636e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:16.743195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857df2bf-8e21-487c-be0a-d84f86636e95 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:16.768557 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1115/2229] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:22:16 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:22:16.773578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c81e6-1131-4c09-b9a1-809f6c3bd84a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:16.784997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3c81e6-1131-4c09-b9a1-809f6c3bd84a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:16.789956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e42a5-6a83-4777-b31e-9d9ba6bd55a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:16.790956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e42a5-6a83-4777-b31e-9d9ba6bd55a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.791242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e42a5-6a83-4777-b31e-9d9ba6bd55a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.791683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e42a5-6a83-4777-b31e-9d9ba6bd55a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.792418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e42a5-6a83-4777-b31e-9d9ba6bd55a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:16.794247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e42a5-6a83-4777-b31e-9d9ba6bd55a2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:16.806175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1115/2230] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:22:16.810742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857df2bf-8e21-487c-be0a-d84f86636e95 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:16.822276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857df2bf-8e21-487c-be0a-d84f86636e95 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:16.827002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ea618-0781-425a-af5e-6aca28d28b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:16.827684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ea618-0781-425a-af5e-6aca28d28b05 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.827947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ea618-0781-425a-af5e-6aca28d28b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.828218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ea618-0781-425a-af5e-6aca28d28b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.828943 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ea618-0781-425a-af5e-6aca28d28b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:16.830588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ea618-0781-425a-af5e-6aca28d28b05 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:16.841294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1116/2231] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:22:16.845268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e42a5-6a83-4777-b31e-9d9ba6bd55a2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:16.855476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e42a5-6a83-4777-b31e-9d9ba6bd55a2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:16.860516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:16.861358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.861634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.861936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] PATH_INFO: `/v3/projects/1c2c4230250047769b6392c3a2f55917/users/09d1e99d9bed47a3a58c4fb0316e7cdd/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.877689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2c4230250047769b6392c3a2f55917, user_id=09d1e99d9bed47a3a58c4fb0316e7cdd, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:16.880398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:16.895934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1116/2232] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:22:16 2022] PUT /identity/v3/projects/1c2c4230250047769b6392c3a2f55917/users/09d1e99d9bed47a3a58c4fb0316e7cdd/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:22:16.906954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a632d5-1c49-4a55-8642-260830be3a53 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:16.907246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a632d5-1c49-4a55-8642-260830be3a53 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:16.907527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a632d5-1c49-4a55-8642-260830be3a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:16.937151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-95a632d5-1c49-4a55-8642-260830be3a53 None None] MFA Rules not processed for user `09d1e99d9bed47a3a58c4fb0316e7cdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:16.938504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-95a632d5-1c49-4a55-8642-260830be3a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:17.007855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1117/2233] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:17.018978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:17.020702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:17.044538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:17.088119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecceab-ec0a-4323-b844-d8d49bf795f3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:17.091915 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:17.092578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:17.092865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:17.093170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:17.103232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:17.104679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:17.114608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1117/2234] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:20.192105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c469b77-15d4-4383-858e-a3f2446ecb65 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:20.192781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c469b77-15d4-4383-858e-a3f2446ecb65 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:20.192781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c469b77-15d4-4383-858e-a3f2446ecb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:20.216887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8c469b77-15d4-4383-858e-a3f2446ecb65 None None] MFA Rules not processed for user `2865db5146784ee188001703d85d8809`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:20.218289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8c469b77-15d4-4383-858e-a3f2446ecb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:20.234455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:20.291588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1118/2235] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:21.924716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:21.926466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:22.033511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cf357-20da-4b58-a166-66ddb0a770de service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:22.042013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cbce5-4339-4b3e-8df4-487a36309268 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:22.042996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cbce5-4339-4b3e-8df4-487a36309268 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:22.043460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cbce5-4339-4b3e-8df4-487a36309268 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:22.043891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cbce5-4339-4b3e-8df4-487a36309268 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:22.046467 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fd5cbce5-4339-4b3e-8df4-487a36309268 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:22.062909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cbce5-4339-4b3e-8df4-487a36309268 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:22.064582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5cbce5-4339-4b3e-8df4-487a36309268 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:22.079244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1118/2236] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:25.466184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec90f-4fbe-4259-ae79-f5313a8a64db None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:25.467172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec90f-4fbe-4259-ae79-f5313a8a64db None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:25.467473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ec90f-4fbe-4259-ae79-f5313a8a64db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:25.491273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-309ec90f-4fbe-4259-ae79-f5313a8a64db None None] MFA Rules not processed for user `8a45cafadb824e08bd9e86fd0d5c4c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:25.492576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-309ec90f-4fbe-4259-ae79-f5313a8a64db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:25.552631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1119/2237] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:30.027638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937111-5b42-4a92-9ecb-ee305d219da2 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.028570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937111-5b42-4a92-9ecb-ee305d219da2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.029068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6937111-5b42-4a92-9ecb-ee305d219da2 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.030770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1119/2238] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:22:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:22:30.039742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a872cb-2bdf-41a7-8266-63aa3352a2a7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.040591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a872cb-2bdf-41a7-8266-63aa3352a2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.040878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a872cb-2bdf-41a7-8266-63aa3352a2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.059476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6937111-5b42-4a92-9ecb-ee305d219da2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:30.065240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-35a872cb-2bdf-41a7-8266-63aa3352a2a7 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:30.066517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-35a872cb-2bdf-41a7-8266-63aa3352a2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:30.143306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6937111-5b42-4a92-9ecb-ee305d219da2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:30.149156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:30.150238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.150525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.150810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] PATH_INFO: `/v3/users/09d1e99d9bed47a3a58c4fb0316e7cdd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.152066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] RBAC: Authorizing `identity:delete_user(user_id=09d1e99d9bed47a3a58c4fb0316e7cdd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:30.153990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:30.179673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1120/2239] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:22:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:30.195660 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a872cb-2bdf-41a7-8266-63aa3352a2a7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:30.197678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-35a872cb-2bdf-41a7-8266-63aa3352a2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:30.234789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09d1e99d9bed47a3a58c4fb0316e7cdd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:30.247217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09d1e99d9bed47a3a58c4fb0316e7cdd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:30.248904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:30.265038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09d1e99d9bed47a3a58c4fb0316e7cdd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:30.266101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09d1e99d9bed47a3a58c4fb0316e7cdd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:30.267300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a89b59c8-21b3-43e8-a4ae-9af68d79d18e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09d1e99d9bed47a3a58c4fb0316e7cdd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:30.283420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a872cb-2bdf-41a7-8266-63aa3352a2a7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:30.288547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:30.289196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.289517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.289830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.291241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1120/2240] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:22:30 2022] DELETE /identity/v3/users/09d1e99d9bed47a3a58c4fb0316e7cdd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:22:30.291724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:30.299152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5e704-a774-4553-bdd6-e2176191ea44 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.299447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5e704-a774-4553-bdd6-e2176191ea44 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.299731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5e704-a774-4553-bdd6-e2176191ea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.306810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:30.308452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:30.325052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ada5e704-a774-4553-bdd6-e2176191ea44 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:30.326416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ada5e704-a774-4553-bdd6-e2176191ea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:30.339829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1121/2241] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:22:30.405857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1121/2242] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:22:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:30.432358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:30.436663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:30.448919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:30.458310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ded325-2f67-4de2-8272-0bfb4b8f3182 None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.459152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ded325-2f67-4de2-8272-0bfb4b8f3182 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.459523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ded325-2f67-4de2-8272-0bfb4b8f3182 None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.470450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1122/2243] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:22:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:22:30.488443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ea0e3-5d23-43cf-89d8-5c25d70ca3e6 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.488727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ea0e3-5d23-43cf-89d8-5c25d70ca3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.489155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ea0e3-5d23-43cf-89d8-5c25d70ca3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.530304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7b9ea0e3-5d23-43cf-89d8-5c25d70ca3e6 None None] MFA Rules not processed for user `1f1fe084c3164e50bf5d904350925f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:30.534056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7b9ea0e3-5d23-43cf-89d8-5c25d70ca3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:30.583615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3563d-be10-475d-943b-dbbd59b0749e service glance] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:30.609882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1123/2244] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:22:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 18 14:22:30.641293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ea0e3-5d23-43cf-89d8-5c25d70ca3e6 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:30.643157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7b9ea0e3-5d23-43cf-89d8-5c25d70ca3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:30.655366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ea0e3-5d23-43cf-89d8-5c25d70ca3e6 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:30.670139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2e1de-1117-4170-8f68-5a1631761905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:30.676875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2e1de-1117-4170-8f68-5a1631761905 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.677577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2e1de-1117-4170-8f68-5a1631761905 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.678207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2e1de-1117-4170-8f68-5a1631761905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.692664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2e1de-1117-4170-8f68-5a1631761905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:30.697691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2e1de-1117-4170-8f68-5a1631761905 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:30.714957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ceb4e-c164-436f-b4e5-bacfde7354b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:30.716545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ceb4e-c164-436f-b4e5-bacfde7354b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.717009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ceb4e-c164-436f-b4e5-bacfde7354b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.717709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ceb4e-c164-436f-b4e5-bacfde7354b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.719165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1124/2245] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:30.735584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ceb4e-c164-436f-b4e5-bacfde7354b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:30.738229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ceb4e-c164-436f-b4e5-bacfde7354b5 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:30.749394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1122/2246] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:30.828589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed1905-b374-4775-9f45-d9bb0de9bb2d None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.829447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed1905-b374-4775-9f45-d9bb0de9bb2d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.829984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed1905-b374-4775-9f45-d9bb0de9bb2d None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.832185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1125/2247] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:22:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:22:30.841046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effef33-1b52-42f3-8492-b4c5a2e5b93c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:30.841463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effef33-1b52-42f3-8492-b4c5a2e5b93c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:30.841902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effef33-1b52-42f3-8492-b4c5a2e5b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:30.888436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5effef33-1b52-42f3-8492-b4c5a2e5b93c None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:30.890035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5effef33-1b52-42f3-8492-b4c5a2e5b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:31.027234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1123/2248] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:22:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:31.041127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed1905-b374-4775-9f45-d9bb0de9bb2d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:31.054052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed1905-b374-4775-9f45-d9bb0de9bb2d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:31.058878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2420f0f-c5ad-4e2f-a743-6a4ccf791b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:31.060613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2420f0f-c5ad-4e2f-a743-6a4ccf791b36 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:31.061894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2420f0f-c5ad-4e2f-a743-6a4ccf791b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:31.062661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2420f0f-c5ad-4e2f-a743-6a4ccf791b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:31.087623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2420f0f-c5ad-4e2f-a743-6a4ccf791b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:31.090227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2420f0f-c5ad-4e2f-a743-6a4ccf791b36 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:31.109155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1126/2249] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:31.679367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effef33-1b52-42f3-8492-b4c5a2e5b93c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:31.681153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5effef33-1b52-42f3-8492-b4c5a2e5b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:31.682749 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2420f0f-c5ad-4e2f-a743-6a4ccf791b36 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:31.694477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2420f0f-c5ad-4e2f-a743-6a4ccf791b36 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:31.697033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effef33-1b52-42f3-8492-b4c5a2e5b93c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:31.704708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:31.715481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:31.716100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:31.716370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:31.716645 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] PATH_INFO: `/v3/projects/1c2c4230250047769b6392c3a2f55917` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:31.717620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:31.717951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:31.718232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:31.720194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2c4230250047769b6392c3a2f55917)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:31.721395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:31.724706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:31.805888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:31.808007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2c4230250047769b6392c3a2f55917', 'request_id': 'req-1749fd26-0383-4320-8c4c-657b46275d04'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:31.818139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2c4230250047769b6392c3a2f55917', 'request_id': 'req-1749fd26-0383-4320-8c4c-657b46275d04'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:31.841369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2c4230250047769b6392c3a2f55917', 'request_id': 'req-1749fd26-0383-4320-8c4c-657b46275d04'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:31.842467 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:31.865120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:31.867487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:31.871715 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] The token cache is being invalidate because project 1c2c4230250047769b6392c3a2f55917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:22:31.872252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:31.873613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1749fd26-0383-4320-8c4c-657b46275d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:22:31.876246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1124/2250] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:22:31 2022] DELETE /identity/v3/projects/1c2c4230250047769b6392c3a2f55917 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:22:31.882426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1127/2251] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:22:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:22:38.526349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6b8db-6b76-4374-8a93-ec7c218d4a78 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:38.526855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6b8db-6b76-4374-8a93-ec7c218d4a78 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:38.526855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6b8db-6b76-4374-8a93-ec7c218d4a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:38.549030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fec6b8db-6b76-4374-8a93-ec7c218d4a78 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:22:38.550628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fec6b8db-6b76-4374-8a93-ec7c218d4a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:38.711014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1125/2252] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:22:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:22:38.742883 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:22:38.744372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:22:38.862837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ac521-40f5-478d-b47a-ebd1fc07e00a service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:22:38.944835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:22:38.945712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:22:38.946005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:22:38.946428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:22:38.957711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:22:38.960119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:22:38.974735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1128/2253] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:22:48.691711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=32,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:22:48.966596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=63 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:13.460079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7a59-7542-457c-ac56-bcfe666c1fc4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:13.460635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7a59-7542-457c-ac56-bcfe666c1fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:13.460635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7a59-7542-457c-ac56-bcfe666c1fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:13.479986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f7ed7a59-7542-457c-ac56-bcfe666c1fc4 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:23:13.481254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f7ed7a59-7542-457c-ac56-bcfe666c1fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:13.490672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1126/2254] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 18 14:23:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:23:13.504567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:13.506842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:13.521636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10163c2-ea60-482e-8328-7bb5c7e1af76 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:13.528197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc838746-4133-469a-8d1f-824c527d4700 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:13.528908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838746-4133-469a-8d1f-824c527d4700 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:13.529183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838746-4133-469a-8d1f-824c527d4700 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:13.529462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc838746-4133-469a-8d1f-824c527d4700 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:13.539551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc838746-4133-469a-8d1f-824c527d4700 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:13.541167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc838746-4133-469a-8d1f-824c527d4700 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:13.549802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1129/2255] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:23:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:23:23.473995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:23.546409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:47.715578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7a59-7542-457c-ac56-bcfe666c1fc4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:47.717188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f7ed7a59-7542-457c-ac56-bcfe666c1fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:47.794000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7a59-7542-457c-ac56-bcfe666c1fc4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:47.798961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:47.799844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:47.800144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:47.800385 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] PATH_INFO: `/v3/users/2865db5146784ee188001703d85d8809` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:47.801610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2865db5146784ee188001703d85d8809)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:47.803662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:47.858161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2865db5146784ee188001703d85d8809'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:47.867655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2865db5146784ee188001703d85d8809'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:47.880532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2865db5146784ee188001703d85d8809'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:47.881383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2865db5146784ee188001703d85d8809'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:47.882081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-4b03def4-e87e-4b2e-be93-a1bead8875cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2865db5146784ee188001703d85d8809'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:47.894462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1127/2256] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:23:47 2022] DELETE /identity/v3/users/2865db5146784ee188001703d85d8809 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:23:48.079942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc838746-4133-469a-8d1f-824c527d4700 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:48.093594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc838746-4133-469a-8d1f-824c527d4700 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:48.156770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:48.157589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.157873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.158150 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] PATH_INFO: `/v3/projects/2bba9fd8306e43b6b1cbc2dd4a789736` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.159362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bba9fd8306e43b6b1cbc2dd4a789736)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:48.161292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:48.210631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bba9fd8306e43b6b1cbc2dd4a789736', 'request_id': 'req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:48.220669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bba9fd8306e43b6b1cbc2dd4a789736', 'request_id': 'req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:48.227664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bba9fd8306e43b6b1cbc2dd4a789736', 'request_id': 'req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:48.238519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] The token cache is being invalidate because project 2bba9fd8306e43b6b1cbc2dd4a789736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:23:48.238808 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:48.239451 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7bae38c5-3af9-4e8a-9de6-cf896f6fe20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:48.241672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1130/2257] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:23:48 2022] DELETE /identity/v3/projects/2bba9fd8306e43b6b1cbc2dd4a789736 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:23:48.247113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d528-0224-4ca6-9b64-ee251b93dfc2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.247402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d528-0224-4ca6-9b64-ee251b93dfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.247708 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17d528-0224-4ca6-9b64-ee251b93dfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.267285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-be17d528-0224-4ca6-9b64-ee251b93dfc2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:23:48.268517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-be17d528-0224-4ca6-9b64-ee251b93dfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:48.336861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1128/2258] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:23:48.345717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15f825-1956-4e60-a6e2-6a2f3479c2ff None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.346086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15f825-1956-4e60-a6e2-6a2f3479c2ff None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.346429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15f825-1956-4e60-a6e2-6a2f3479c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.371041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8c15f825-1956-4e60-a6e2-6a2f3479c2ff None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:23:48.372278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8c15f825-1956-4e60-a6e2-6a2f3479c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:48.381022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1131/2259] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:23:48.385726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17d528-0224-4ca6-9b64-ee251b93dfc2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:48.399565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17d528-0224-4ca6-9b64-ee251b93dfc2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:48.405273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:48.406020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.406302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.406574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.407363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:23:48.407606 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:48.409276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:48.421611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1129/2260] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:23:48.426349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15f825-1956-4e60-a6e2-6a2f3479c2ff None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:48.437123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15f825-1956-4e60-a6e2-6a2f3479c2ff None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:48.443293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e218c71-d074-410e-9661-9c19897eff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:48.444045 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e218c71-d074-410e-9661-9c19897eff15 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.444434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e218c71-d074-410e-9661-9c19897eff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.444710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e218c71-d074-410e-9661-9c19897eff15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.445405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e218c71-d074-410e-9661-9c19897eff15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:48.447359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e218c71-d074-410e-9661-9c19897eff15 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:48.477042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1132/2261] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:23:48 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:23:48.481319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:48.492717 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d4d5-3c5a-4894-8694-d2b62e52dd49 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:48.497031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:48.558888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75af3-4851-4aa8-a9ed-4f652fbbb08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:48.559940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75af3-4851-4aa8-a9ed-4f652fbbb08b admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.560432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75af3-4851-4aa8-a9ed-4f652fbbb08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.560931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be75af3-4851-4aa8-a9ed-4f652fbbb08b admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.562070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be75af3-4851-4aa8-a9ed-4f652fbbb08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:48.564915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be75af3-4851-4aa8-a9ed-4f652fbbb08b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:48.594881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1130/2262] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:23:48 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:23:48.599249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e218c71-d074-410e-9661-9c19897eff15 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:48.610234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e218c71-d074-410e-9661-9c19897eff15 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:48.615431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1078a-85b5-4877-9d84-c4f7dea1d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:48.616264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1078a-85b5-4877-9d84-c4f7dea1d57d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.616536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1078a-85b5-4877-9d84-c4f7dea1d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.616818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1078a-85b5-4877-9d84-c4f7dea1d57d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.617683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1078a-85b5-4877-9d84-c4f7dea1d57d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:48.619331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1078a-85b5-4877-9d84-c4f7dea1d57d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:48.634126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1133/2263] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:23:48.635397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75af3-4851-4aa8-a9ed-4f652fbbb08b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:48.646477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be75af3-4851-4aa8-a9ed-4f652fbbb08b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:48.652394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386998d-3736-4315-9742-7732f6fd32b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:48.653087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386998d-3736-4315-9742-7732f6fd32b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.653373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386998d-3736-4315-9742-7732f6fd32b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.653677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386998d-3736-4315-9742-7732f6fd32b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.654415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386998d-3736-4315-9742-7732f6fd32b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:48.656054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386998d-3736-4315-9742-7732f6fd32b3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:48.667126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1131/2264] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:23:48.671381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1078a-85b5-4877-9d84-c4f7dea1d57d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:48.681759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1078a-85b5-4877-9d84-c4f7dea1d57d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:48.687001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:48.687914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.688201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.688482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] PATH_INFO: `/v3/projects/7168acdb78a7427294942ee2456e3bc6/users/af107a7fe5074935ad32d7362a14e6df/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.703771 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7168acdb78a7427294942ee2456e3bc6, user_id=af107a7fe5074935ad32d7362a14e6df, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:48.706494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:48.722423 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1134/2265] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:23:48 2022] PUT /identity/v3/projects/7168acdb78a7427294942ee2456e3bc6/users/af107a7fe5074935ad32d7362a14e6df/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:23:48.733144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355455-8c93-496f-a953-4cbe33b8ca49 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.733778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355455-8c93-496f-a953-4cbe33b8ca49 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.733778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99355455-8c93-496f-a953-4cbe33b8ca49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.763473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-99355455-8c93-496f-a953-4cbe33b8ca49 None None] MFA Rules not processed for user `af107a7fe5074935ad32d7362a14e6df`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:23:48.764726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-99355455-8c93-496f-a953-4cbe33b8ca49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:48.837740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1132/2266] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:23:48.849617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:48.851777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:48.884211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:48.914715 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf34e2-1713-4039-b251-fd84796ff210 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:48.918885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d283517-8d74-48c7-9310-571372617ff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:48.919474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d283517-8d74-48c7-9310-571372617ff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:48.919757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d283517-8d74-48c7-9310-571372617ff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:48.920033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d283517-8d74-48c7-9310-571372617ff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:48.932508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d283517-8d74-48c7-9310-571372617ff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:48.934855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d283517-8d74-48c7-9310-571372617ff7 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:48.946658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1135/2267] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:57.126552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99355455-8c93-496f-a953-4cbe33b8ca49 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:57.197783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99355455-8c93-496f-a953-4cbe33b8ca49 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:57.202670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:57.203902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:57.204190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:57.204468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] PATH_INFO: `/v3/users/af107a7fe5074935ad32d7362a14e6df` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:57.205884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af107a7fe5074935ad32d7362a14e6df)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:57.208653 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:57.230410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:23:57.280064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af107a7fe5074935ad32d7362a14e6df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:57.293199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af107a7fe5074935ad32d7362a14e6df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:57.320778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af107a7fe5074935ad32d7362a14e6df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:57.326226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af107a7fe5074935ad32d7362a14e6df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:57.327044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af107a7fe5074935ad32d7362a14e6df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:57.358288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1133/2268] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:23:57 2022] DELETE /identity/v3/users/af107a7fe5074935ad32d7362a14e6df => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:23:57.363114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29cbe9-61da-48f8-bb2c-f434cf64e44f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:57.363686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29cbe9-61da-48f8-bb2c-f434cf64e44f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:57.363686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29cbe9-61da-48f8-bb2c-f434cf64e44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:57.389783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0b29cbe9-61da-48f8-bb2c-f434cf64e44f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:23:57.391523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0b29cbe9-61da-48f8-bb2c-f434cf64e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:57.498141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1136/2269] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:23:57.508393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:57.510888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:57.592523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b6a06-e299-45f0-9cc1-6fd13fa246a8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:57.596472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:57.597137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:57.597552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:57.598038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:57.609344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:57.610936 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:57.622463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1134/2270] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:23:57.997922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29cbe9-61da-48f8-bb2c-f434cf64e44f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:23:57.999659 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0b29cbe9-61da-48f8-bb2c-f434cf64e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:23:58.024188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29cbe9-61da-48f8-bb2c-f434cf64e44f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:23:58.032382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:23:58.033012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:23:58.033293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:23:58.033720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] PATH_INFO: `/v3/projects/7168acdb78a7427294942ee2456e3bc6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:23:58.035086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7168acdb78a7427294942ee2456e3bc6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:23:58.037505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:23:58.157566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7168acdb78a7427294942ee2456e3bc6', 'request_id': 'req-027902b5-92ac-47f2-ad94-fd5367653a10'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:58.172841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7168acdb78a7427294942ee2456e3bc6', 'request_id': 'req-027902b5-92ac-47f2-ad94-fd5367653a10'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:58.187766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7168acdb78a7427294942ee2456e3bc6', 'request_id': 'req-027902b5-92ac-47f2-ad94-fd5367653a10'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:58.203222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] The token cache is being invalidate because project 7168acdb78a7427294942ee2456e3bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:23:58.203587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:58.204396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:23:58.206730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1137/2271] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:23:57 2022] DELETE /identity/v3/projects/7168acdb78a7427294942ee2456e3bc6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:24:02.561801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:02.563289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:02.574841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b0839-1b98-478c-a1c5-539eafd02dd5 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:02.640240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b90322-25d4-4c3c-91db-49d2fe4bbf68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:02.640913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b90322-25d4-4c3c-91db-49d2fe4bbf68 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:02.641197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b90322-25d4-4c3c-91db-49d2fe4bbf68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:02.641471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b90322-25d4-4c3c-91db-49d2fe4bbf68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:02.643415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-21b90322-25d4-4c3c-91db-49d2fe4bbf68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:02.696886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:02.723344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b90322-25d4-4c3c-91db-49d2fe4bbf68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:02.724828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b90322-25d4-4c3c-91db-49d2fe4bbf68 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:02.736715 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1135/2272] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:24:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:24:06.087563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:06.089330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:06.105309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027902b5-92ac-47f2-ad94-fd5367653a10 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:06.109125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0618d2-bb74-472b-bda7-a3ab34e463c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:06.109845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0618d2-bb74-472b-bda7-a3ab34e463c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:06.110111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0618d2-bb74-472b-bda7-a3ab34e463c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:06.110440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0618d2-bb74-472b-bda7-a3ab34e463c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:06.112511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cb0618d2-bb74-472b-bda7-a3ab34e463c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:06.127499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0618d2-bb74-472b-bda7-a3ab34e463c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:06.129001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0618d2-bb74-472b-bda7-a3ab34e463c3 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:06.141296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1138/2273] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:24:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:24:12.731256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=10 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:15.587249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc71ae-630e-430e-8f3b-3edd9e6426c0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:15.588263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc71ae-630e-430e-8f3b-3edd9e6426c0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:15.588545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc71ae-630e-430e-8f3b-3edd9e6426c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:15.622548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d1fc71ae-630e-430e-8f3b-3edd9e6426c0 None None] MFA Rules not processed for user `a0210f69010b4529822bcdac1bdf8b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:24:15.623851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d1fc71ae-630e-430e-8f3b-3edd9e6426c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:15.687241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1136/2274] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:24:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:24:16.135564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:17.769282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee057f-7368-41cf-8b8d-2192ff86bb8c None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:17.769990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee057f-7368-41cf-8b8d-2192ff86bb8c None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:17.769990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee057f-7368-41cf-8b8d-2192ff86bb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:17.789410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f7ee057f-7368-41cf-8b8d-2192ff86bb8c None None] MFA Rules not processed for user `3aab24bab8d74b7aa5a20e7b2b89b5d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:24:17.790952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f7ee057f-7368-41cf-8b8d-2192ff86bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:17.847713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1139/2275] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:24:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:24:17.859151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc71ae-630e-430e-8f3b-3edd9e6426c0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:17.860747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d1fc71ae-630e-430e-8f3b-3edd9e6426c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:17.877367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc71ae-630e-430e-8f3b-3edd9e6426c0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:17.883140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:17.883804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:17.884083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:17.884360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:17.886736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:17.899241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:17.901197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:17.910986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1137/2276] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:24:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:24:21.672302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee057f-7368-41cf-8b8d-2192ff86bb8c None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:21.674796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f7ee057f-7368-41cf-8b8d-2192ff86bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:21.694745 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee057f-7368-41cf-8b8d-2192ff86bb8c None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:21.701602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab072b17-2927-45c5-8f9d-471daaaa7824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:21.702585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab072b17-2927-45c5-8f9d-471daaaa7824 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:21.702878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab072b17-2927-45c5-8f9d-471daaaa7824 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:21.703224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab072b17-2927-45c5-8f9d-471daaaa7824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:21.704977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ab072b17-2927-45c5-8f9d-471daaaa7824 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:21.717417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab072b17-2927-45c5-8f9d-471daaaa7824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:21.719124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab072b17-2927-45c5-8f9d-471daaaa7824 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:21.734621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1140/2277] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:24:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:24:22.230953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:22.232640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:22.302810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364e49a-3574-401f-9368-292cd4c8a4cc service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:22.307152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:22.307916 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:22.308204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:22.308480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:22.310424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:22.375614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:22.377078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:22.388987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1138/2278] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:24:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:24:26.183950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc1ed9-5c9c-43cd-a478-74ed4ac4e340 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:26.184558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc1ed9-5c9c-43cd-a478-74ed4ac4e340 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:26.184558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc1ed9-5c9c-43cd-a478-74ed4ac4e340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:26.205046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e3bc1ed9-5c9c-43cd-a478-74ed4ac4e340 None None] MFA Rules not processed for user `8a45cafadb824e08bd9e86fd0d5c4c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:24:26.206385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e3bc1ed9-5c9c-43cd-a478-74ed4ac4e340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:26.215390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1141/2279] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:24:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:24:32.382311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=79 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:33.742400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:33.753842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53479ae-7e23-494b-9baa-f8b5916be5d6 service glance] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:33.758344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776cf554-135e-4792-9eda-da32f0e0d3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:33.759003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf554-135e-4792-9eda-da32f0e0d3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:33.759288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf554-135e-4792-9eda-da32f0e0d3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:33.759527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776cf554-135e-4792-9eda-da32f0e0d3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:33.761617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-776cf554-135e-4792-9eda-da32f0e0d3a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:33.772830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cf554-135e-4792-9eda-da32f0e0d3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:33.774359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776cf554-135e-4792-9eda-da32f0e0d3a3 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:33.784274 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1139/2280] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:24:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:24:36.199161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=34 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:43.779223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:46.988128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc1ed9-5c9c-43cd-a478-74ed4ac4e340 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:47.002293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc1ed9-5c9c-43cd-a478-74ed4ac4e340 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:47.009034 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:47.009737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:47.010128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:47.010333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:47.012329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:47.026476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:47.028161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:47.041180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1142/2281] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:24:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:24:47.795715 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18da30-9386-44dd-b57c-8504ce38c75f None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:47.796279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18da30-9386-44dd-b57c-8504ce38c75f None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:47.796279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18da30-9386-44dd-b57c-8504ce38c75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:47.817741 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1c18da30-9386-44dd-b57c-8504ce38c75f None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:24:47.819033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1c18da30-9386-44dd-b57c-8504ce38c75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:47.886486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1140/2282] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 18 14:24:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:24:47.948217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:47.950030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:48.025686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc583c0-0d5b-4088-8c5f-ec89d3c28512 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:48.033032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b634b4-f583-4243-bb81-ab222a5fe331 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:48.033765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b634b4-f583-4243-bb81-ab222a5fe331 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:48.034429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b634b4-f583-4243-bb81-ab222a5fe331 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:48.034842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b634b4-f583-4243-bb81-ab222a5fe331 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:48.048842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b634b4-f583-4243-bb81-ab222a5fe331 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:48.050429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b634b4-f583-4243-bb81-ab222a5fe331 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:48.059726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1143/2283] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:24:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:24:55.438211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18da30-9386-44dd-b57c-8504ce38c75f None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:55.439719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1c18da30-9386-44dd-b57c-8504ce38c75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:55.524882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18da30-9386-44dd-b57c-8504ce38c75f None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:55.529605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:55.530547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:55.530827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:55.531106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] PATH_INFO: `/v3/users/a9057278ae1c42a5a2802a3be11828df` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:55.532466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9057278ae1c42a5a2802a3be11828df)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:55.534584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:55.587679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9057278ae1c42a5a2802a3be11828df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:55.599847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9057278ae1c42a5a2802a3be11828df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:55.623485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9057278ae1c42a5a2802a3be11828df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:55.624359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9057278ae1c42a5a2802a3be11828df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:55.625025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9057278ae1c42a5a2802a3be11828df'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:55.643391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1141/2284] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:24:55 2022] DELETE /identity/v3/users/a9057278ae1c42a5a2802a3be11828df => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:24:55.872586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b634b4-f583-4243-bb81-ab222a5fe331 service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:55.885082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b634b4-f583-4243-bb81-ab222a5fe331 service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:55.956977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:55.958254 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:55.958738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:55.959028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] PATH_INFO: `/v3/projects/af54954fbeeb4772ad57290fceb3b177` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:55.960580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af54954fbeeb4772ad57290fceb3b177)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:55.962627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:56.015209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af54954fbeeb4772ad57290fceb3b177', 'request_id': 'req-bd738342-477e-4ec3-9e80-4ce53b6a0c66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.024713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af54954fbeeb4772ad57290fceb3b177', 'request_id': 'req-bd738342-477e-4ec3-9e80-4ce53b6a0c66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.032115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af54954fbeeb4772ad57290fceb3b177', 'request_id': 'req-bd738342-477e-4ec3-9e80-4ce53b6a0c66'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.048532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] The token cache is being invalidate because project af54954fbeeb4772ad57290fceb3b177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:24:56.049108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.049506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.051191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1144/2285] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:24:55 2022] DELETE /identity/v3/projects/af54954fbeeb4772ad57290fceb3b177 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:24:56.055424 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:56.057067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:56.072622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5302e0b-5d5e-4544-8d7a-73e2456c33fc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:56.104839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:56.157115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:56.158658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:56.159145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:56.159626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] PATH_INFO: `/v3/users/18183f4a30774996baca9a555b84a007` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:56.161406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18183f4a30774996baca9a555b84a007)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:56.163452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:56.218020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18183f4a30774996baca9a555b84a007'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.229767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18183f4a30774996baca9a555b84a007'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.251295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18183f4a30774996baca9a555b84a007'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.252068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18183f4a30774996baca9a555b84a007'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.252752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18183f4a30774996baca9a555b84a007'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.271115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1142/2286] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:24:56 2022] DELETE /identity/v3/users/18183f4a30774996baca9a555b84a007 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:24:56.487701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:24:56.489762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:24:56.509504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd738342-477e-4ec3-9e80-4ce53b6a0c66 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:24:56.513682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:24:56.589859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:24:56.590773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:24:56.591057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:24:56.591340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] PATH_INFO: `/v3/projects/46ae3663cb1f4af7b69b88d1996d757f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:24:56.592681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] RBAC: Authorizing `identity:delete_project(project_id=46ae3663cb1f4af7b69b88d1996d757f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:24:56.595095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:24:56.650126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46ae3663cb1f4af7b69b88d1996d757f', 'request_id': 'req-cd31f10b-e1ea-4118-be20-47df2076b66f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.660614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46ae3663cb1f4af7b69b88d1996d757f', 'request_id': 'req-cd31f10b-e1ea-4118-be20-47df2076b66f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.669184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46ae3663cb1f4af7b69b88d1996d757f', 'request_id': 'req-cd31f10b-e1ea-4118-be20-47df2076b66f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.684384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] The token cache is being invalidate because project 46ae3663cb1f4af7b69b88d1996d757f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:24:56.684955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.685974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cd31f10b-e1ea-4118-be20-47df2076b66f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:24:56.687700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1145/2287] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:24:56 2022] DELETE /identity/v3/projects/46ae3663cb1f4af7b69b88d1996d757f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:06.265058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:06.679144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:08.490015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:08.491788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:08.553363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5749a56-c7ce-493a-be72-1c63baf8014f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:08.619989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:08.620580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:08.620866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:08.621142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:08.623272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:08.694571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:08.696038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:08.705624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1143/2288] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:18.701000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=68 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:22.323078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a022d-12bb-4ab9-8f5e-076cf456fdce None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:22.323581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a022d-12bb-4ab9-8f5e-076cf456fdce None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:22.323581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a022d-12bb-4ab9-8f5e-076cf456fdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:22.345446 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9a0a022d-12bb-4ab9-8f5e-076cf456fdce None None] MFA Rules not processed for user `5ca8be8093b547ee953ea646f90fdcae`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:22.347124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9a0a022d-12bb-4ab9-8f5e-076cf456fdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:22.357128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1146/2289] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:23.049180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:23.051502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:23.119961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c543d-8276-430b-af1e-46d67fe1f75f service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:23.124073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:23.124848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:23.125582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:23.125889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:23.127927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:23.138377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:23.139820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:23.149070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1144/2290] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:23.495702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a022d-12bb-4ab9-8f5e-076cf456fdce None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:23.497341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9a0a022d-12bb-4ab9-8f5e-076cf456fdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:23.591421 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a022d-12bb-4ab9-8f5e-076cf456fdce None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:23.596787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:23.597487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:23.597791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:23.598030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:23.599948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:23.612503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:23.614165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:23.625272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1147/2291] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:26.999321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:27.001721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:27.181043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031efa59-a42d-4a75-98fa-7328eb2bde1e service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:27.188168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:27.188980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:27.189369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:27.189809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:27.192819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:27.209637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:27.211619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:27.225443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1145/2292] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:25:33.210523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:33.211960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:33.224779 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bd54a-9868-4806-928e-643e18c52ce7 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:33.228417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:33.229314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:33.229595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:33.229915 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:33.231753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:33.306360 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:33.308385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:33.319049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1148/2293] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:35.124528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:35.143039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366dcbe2-f909-4ea2-9ae6-4b884b1f0a90 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:35.162325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:35.163396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:35.164042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:35.164682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:35.168330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:35.200003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:35.204048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:35.243698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1146/2294] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:37.212472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:37.230160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d8380-5405-4c23-af6e-0bb9f0adb097 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:37.236544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:37.237442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:37.237911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:37.238355 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:37.240333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:37.262025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:37.264342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:37.280068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1149/2295] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:37.290727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:37.292731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:37.327097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9998a-ecf7-4dd4-8b32-e5d0ed5b20c8 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:37.332769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:37.333880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:37.334254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:37.334758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] PATH_INFO: `/v3/users/8a45cafadb824e08bd9e86fd0d5c4c71` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:37.336506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a45cafadb824e08bd9e86fd0d5c4c71)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:37.339232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:37.410774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a45cafadb824e08bd9e86fd0d5c4c71'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.419429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a45cafadb824e08bd9e86fd0d5c4c71'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.438652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:37.443237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a45cafadb824e08bd9e86fd0d5c4c71'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.444482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a45cafadb824e08bd9e86fd0d5c4c71'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.445815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-29f32917-34e8-48d5-8098-52953358b92f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a45cafadb824e08bd9e86fd0d5c4c71'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.467355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1147/2296] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:37 2022] DELETE /identity/v3/users/8a45cafadb824e08bd9e86fd0d5c4c71 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:37.672260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:37.695830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0518e7-45fe-472d-ac9c-32f516cdc363 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:37.807263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:37.808379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:37.808665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:37.808943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] PATH_INFO: `/v3/projects/38cf167c2fc74ac09d6f6d6f81829cda` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:37.810222 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=38cf167c2fc74ac09d6f6d6f81829cda)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:37.812170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:37.823559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:37.871827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38cf167c2fc74ac09d6f6d6f81829cda', 'request_id': 'req-6ac1719e-418d-4a8c-b301-18897d4675ed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.881615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38cf167c2fc74ac09d6f6d6f81829cda', 'request_id': 'req-6ac1719e-418d-4a8c-b301-18897d4675ed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.888631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38cf167c2fc74ac09d6f6d6f81829cda', 'request_id': 'req-6ac1719e-418d-4a8c-b301-18897d4675ed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.902206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] The token cache is being invalidate because project 38cf167c2fc74ac09d6f6d6f81829cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:37.902496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.903164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-6ac1719e-418d-4a8c-b301-18897d4675ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:37.905250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1150/2297] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:37 2022] DELETE /identity/v3/projects/38cf167c2fc74ac09d6f6d6f81829cda => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:37.911271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df15d-4abc-4a22-b683-e40085ea659a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:37.911517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df15d-4abc-4a22-b683-e40085ea659a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:37.911801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df15d-4abc-4a22-b683-e40085ea659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:37.931197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1a5df15d-4abc-4a22-b683-e40085ea659a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:37.932526 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1a5df15d-4abc-4a22-b683-e40085ea659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:38.017801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1148/2298] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:38.026108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a851ad-2456-4a3e-8006-1f0015a8bbfc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.026400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a851ad-2456-4a3e-8006-1f0015a8bbfc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.026684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a851ad-2456-4a3e-8006-1f0015a8bbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.054784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-c0a851ad-2456-4a3e-8006-1f0015a8bbfc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:38.056457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c0a851ad-2456-4a3e-8006-1f0015a8bbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:38.066563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1151/2299] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:38.074095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5df15d-4abc-4a22-b683-e40085ea659a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.084446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5df15d-4abc-4a22-b683-e40085ea659a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.093727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.095778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.096248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.096741 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.097945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:25:38.098436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.100900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:38.119102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1149/2300] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:25:38.123172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a851ad-2456-4a3e-8006-1f0015a8bbfc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.134612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a851ad-2456-4a3e-8006-1f0015a8bbfc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.140677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7982-d6c1-4c34-a4dc-1362443074ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.140677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7982-d6c1-4c34-a4dc-1362443074ec admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.141093 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7982-d6c1-4c34-a4dc-1362443074ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.141470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7982-d6c1-4c34-a4dc-1362443074ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.142582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e7982-d6c1-4c34-a4dc-1362443074ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.144697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e7982-d6c1-4c34-a4dc-1362443074ec admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:38.180914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1152/2301] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:38 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:38.182646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.193124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3884fe-14e6-44c0-b9b6-2312c70267d5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.339930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36cc7d-238e-4824-af0b-06bca7ef9c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.340599 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cc7d-238e-4824-af0b-06bca7ef9c69 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.340819 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cc7d-238e-4824-af0b-06bca7ef9c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.341271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36cc7d-238e-4824-af0b-06bca7ef9c69 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.342059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36cc7d-238e-4824-af0b-06bca7ef9c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.344253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36cc7d-238e-4824-af0b-06bca7ef9c69 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:38.373188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1150/2302] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:38 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:38.377898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7982-d6c1-4c34-a4dc-1362443074ec admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.390501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7982-d6c1-4c34-a4dc-1362443074ec admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.396063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e882f92-a3d5-49f7-8ca6-61495eff5a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.397140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882f92-a3d5-49f7-8ca6-61495eff5a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.397581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882f92-a3d5-49f7-8ca6-61495eff5a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.398035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e882f92-a3d5-49f7-8ca6-61495eff5a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.398960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e882f92-a3d5-49f7-8ca6-61495eff5a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.400758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e882f92-a3d5-49f7-8ca6-61495eff5a43 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:38.422460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1153/2303] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:25:38.426366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36cc7d-238e-4824-af0b-06bca7ef9c69 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.439074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36cc7d-238e-4824-af0b-06bca7ef9c69 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.444123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b17ec-6dec-4f5a-8c00-9dcb6d97e708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.444899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b17ec-6dec-4f5a-8c00-9dcb6d97e708 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.445266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b17ec-6dec-4f5a-8c00-9dcb6d97e708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.445540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b17ec-6dec-4f5a-8c00-9dcb6d97e708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.446320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b17ec-6dec-4f5a-8c00-9dcb6d97e708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.447949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b17ec-6dec-4f5a-8c00-9dcb6d97e708 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:38.459972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1151/2304] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:38.464261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e882f92-a3d5-49f7-8ca6-61495eff5a43 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.477052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e882f92-a3d5-49f7-8ca6-61495eff5a43 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.482209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ce670-04bf-41fe-b961-f7f3899a96b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.483124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ce670-04bf-41fe-b961-f7f3899a96b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.483412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ce670-04bf-41fe-b961-f7f3899a96b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.483696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ce670-04bf-41fe-b961-f7f3899a96b1 admin admin] PATH_INFO: `/v3/projects/e5b1dbdea4c344828aa8ab0392f1b71c/users/1e94bb4bf00a4f0093232916a85439a0/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.500456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ce670-04bf-41fe-b961-f7f3899a96b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5b1dbdea4c344828aa8ab0392f1b71c, user_id=1e94bb4bf00a4f0093232916a85439a0, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.503441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ce670-04bf-41fe-b961-f7f3899a96b1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:38.521773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1154/2305] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:38 2022] PUT /identity/v3/projects/e5b1dbdea4c344828aa8ab0392f1b71c/users/1e94bb4bf00a4f0093232916a85439a0/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:25:38.532822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8363d0-4873-44e3-8677-688820ec77df None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.532822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8363d0-4873-44e3-8677-688820ec77df None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.533142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8363d0-4873-44e3-8677-688820ec77df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.566419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-be8363d0-4873-44e3-8677-688820ec77df None None] MFA Rules not processed for user `1e94bb4bf00a4f0093232916a85439a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:38.567647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-be8363d0-4873-44e3-8677-688820ec77df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:38.620283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:38.662317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1152/2306] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:38.666988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ce670-04bf-41fe-b961-f7f3899a96b1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.759220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ce670-04bf-41fe-b961-f7f3899a96b1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.768567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e22c55-34ad-47a5-8a8d-99862ef306d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.769982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e22c55-34ad-47a5-8a8d-99862ef306d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.770550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e22c55-34ad-47a5-8a8d-99862ef306d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.771060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e22c55-34ad-47a5-8a8d-99862ef306d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.772394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e22c55-34ad-47a5-8a8d-99862ef306d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.776790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e22c55-34ad-47a5-8a8d-99862ef306d8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:38.812389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1155/2307] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:38 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:38.818207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8363d0-4873-44e3-8677-688820ec77df None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.838800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8363d0-4873-44e3-8677-688820ec77df None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.920433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fae14-7adb-4ead-8a71-ca710678b54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.921137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fae14-7adb-4ead-8a71-ca710678b54e admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.921456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fae14-7adb-4ead-8a71-ca710678b54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.921728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85fae14-7adb-4ead-8a71-ca710678b54e admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.923634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fae14-7adb-4ead-8a71-ca710678b54e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.925143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85fae14-7adb-4ead-8a71-ca710678b54e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:38.958230 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1153/2308] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:38 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:38.959449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e22c55-34ad-47a5-8a8d-99862ef306d8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:38.973848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e22c55-34ad-47a5-8a8d-99862ef306d8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:38.980382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46e0fc-5613-44f4-a233-365932822229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:38.981223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46e0fc-5613-44f4-a233-365932822229 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:38.981521 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46e0fc-5613-44f4-a233-365932822229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:38.981877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46e0fc-5613-44f4-a233-365932822229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:38.982848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46e0fc-5613-44f4-a233-365932822229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:38.984427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46e0fc-5613-44f4-a233-365932822229 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:39.003125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fae14-7adb-4ead-8a71-ca710678b54e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:39.003570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1156/2309] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:39.016977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85fae14-7adb-4ead-8a71-ca710678b54e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:39.023621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff444-215b-4562-a2fe-fea819493be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:39.024556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff444-215b-4562-a2fe-fea819493be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:39.025126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff444-215b-4562-a2fe-fea819493be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:39.025414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ff444-215b-4562-a2fe-fea819493be5 admin admin] PATH_INFO: `/v3/projects/c63588e3ee834ab29341df90281ee2a1/users/1a2ae69bfa224dce82887632dac03de7/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:39.058511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ff444-215b-4562-a2fe-fea819493be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63588e3ee834ab29341df90281ee2a1, user_id=1a2ae69bfa224dce82887632dac03de7, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:39.066096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ff444-215b-4562-a2fe-fea819493be5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:39.096941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1154/2310] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:39 2022] PUT /identity/v3/projects/c63588e3ee834ab29341df90281ee2a1/users/1a2ae69bfa224dce82887632dac03de7/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:25:39.102007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46e0fc-5613-44f4-a233-365932822229 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:39.114147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:39.125020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46e0fc-5613-44f4-a233-365932822229 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:39.285299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039a23f-251a-445d-9adb-56e57bd453cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:39.286039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a23f-251a-445d-9adb-56e57bd453cd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:39.286320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a23f-251a-445d-9adb-56e57bd453cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:39.286598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039a23f-251a-445d-9adb-56e57bd453cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:39.287420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039a23f-251a-445d-9adb-56e57bd453cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:39.289039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039a23f-251a-445d-9adb-56e57bd453cd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:39.308680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1157/2311] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:39.309919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff444-215b-4562-a2fe-fea819493be5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:39.321633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ff444-215b-4562-a2fe-fea819493be5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:39.327219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:39.328767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:39.329089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:39.329406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] PATH_INFO: `/v3/projects/c63588e3ee834ab29341df90281ee2a1/users/1a2ae69bfa224dce82887632dac03de7/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:39.331655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63588e3ee834ab29341df90281ee2a1, user_id=1a2ae69bfa224dce82887632dac03de7, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:39.335302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:39.351246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1155/2312] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:39 2022] PUT /identity/v3/projects/c63588e3ee834ab29341df90281ee2a1/users/1a2ae69bfa224dce82887632dac03de7/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:25:39.362372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889d31a-9d7c-4807-b9bf-0688cf280fd9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:39.362731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889d31a-9d7c-4807-b9bf-0688cf280fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:39.363022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889d31a-9d7c-4807-b9bf-0688cf280fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:39.393356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6889d31a-9d7c-4807-b9bf-0688cf280fd9 None None] MFA Rules not processed for user `1a2ae69bfa224dce82887632dac03de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:39.394636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6889d31a-9d7c-4807-b9bf-0688cf280fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:39.484031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1158/2313] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:39.499484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:39.501846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:39.566742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9026d0f-526a-49c8-9e0a-40ccfac6ec08 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:39.570609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ebaf8-1d50-43a4-a9fb-ef55561a8cd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:39.571395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ebaf8-1d50-43a4-a9fb-ef55561a8cd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:39.571746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ebaf8-1d50-43a4-a9fb-ef55561a8cd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:39.572083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ebaf8-1d50-43a4-a9fb-ef55561a8cd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:39.582877 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ebaf8-1d50-43a4-a9fb-ef55561a8cd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:39.584842 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ebaf8-1d50-43a4-a9fb-ef55561a8cd5 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:39.597449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1156/2314] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:25:40.119490 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889d31a-9d7c-4807-b9bf-0688cf280fd9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:40.226982 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889d31a-9d7c-4807-b9bf-0688cf280fd9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:40.248644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:40.249707 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:40.249962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:40.250324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] PATH_INFO: `/v3/users/1e94bb4bf00a4f0093232916a85439a0` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:40.252078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e94bb4bf00a4f0093232916a85439a0)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:40.254367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:40.318955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:40.351619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e94bb4bf00a4f0093232916a85439a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:40.365042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e94bb4bf00a4f0093232916a85439a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:40.385261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e94bb4bf00a4f0093232916a85439a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:40.386902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e94bb4bf00a4f0093232916a85439a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:40.388123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e94bb4bf00a4f0093232916a85439a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:40.411528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1159/2315] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:40 2022] DELETE /identity/v3/users/1e94bb4bf00a4f0093232916a85439a0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:40.416655 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e330-8326-4794-8b2b-7b77c7d42192 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:40.417581 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e330-8326-4794-8b2b-7b77c7d42192 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:40.417581 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e330-8326-4794-8b2b-7b77c7d42192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:40.439991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-4647e330-8326-4794-8b2b-7b77c7d42192 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:40.441432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4647e330-8326-4794-8b2b-7b77c7d42192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:40.485576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:40.541157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1157/2316] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:40.583602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:40.587021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:40.769980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3b9af-a5d6-40b1-8bf2-e656afa63475 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:40.770334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ae44e-c340-4079-8a88-b0dd9084a10c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:40.770617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ae44e-c340-4079-8a88-b0dd9084a10c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:40.770895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ae44e-c340-4079-8a88-b0dd9084a10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:40.777269 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15018d73-801f-4c7c-b280-a19e7721528d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:40.778145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15018d73-801f-4c7c-b280-a19e7721528d service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:40.779172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15018d73-801f-4c7c-b280-a19e7721528d service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:40.779627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15018d73-801f-4c7c-b280-a19e7721528d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:40.792538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15018d73-801f-4c7c-b280-a19e7721528d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:40.794316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f30ae44e-c340-4079-8a88-b0dd9084a10c None None] MFA Rules not processed for user `a0210f69010b4529822bcdac1bdf8b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:40.794807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15018d73-801f-4c7c-b280-a19e7721528d service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:40.795905 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f30ae44e-c340-4079-8a88-b0dd9084a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:40.825382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1160/2317] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:40.892765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1158/2318] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:25:41.103701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cee9eb-cdcc-412d-96bc-b080c12ce87a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:41.104797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cee9eb-cdcc-412d-96bc-b080c12ce87a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:41.105584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cee9eb-cdcc-412d-96bc-b080c12ce87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:41.135620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-06cee9eb-cdcc-412d-96bc-b080c12ce87a None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:41.137291 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-06cee9eb-cdcc-412d-96bc-b080c12ce87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:41.253112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1161/2319] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:25:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:41.268591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ae44e-c340-4079-8a88-b0dd9084a10c None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:41.270190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f30ae44e-c340-4079-8a88-b0dd9084a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:41.380690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ae44e-c340-4079-8a88-b0dd9084a10c None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:41.392661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c11c3-160d-4831-b777-521d4648f19e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:41.393349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c11c3-160d-4831-b777-521d4648f19e service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:41.393636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c11c3-160d-4831-b777-521d4648f19e service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:41.394346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c11c3-160d-4831-b777-521d4648f19e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:41.404037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c11c3-160d-4831-b777-521d4648f19e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:41.404839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c11c3-160d-4831-b777-521d4648f19e service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:41.416402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1159/2320] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:25:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:25:41.506473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cee9eb-cdcc-412d-96bc-b080c12ce87a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:41.508897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-06cee9eb-cdcc-412d-96bc-b080c12ce87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:41.536664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cee9eb-cdcc-412d-96bc-b080c12ce87a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:41.550838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:41.552675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:41.553035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:41.553606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] PATH_INFO: `/v3/projects/e5b1dbdea4c344828aa8ab0392f1b71c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:41.556408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5b1dbdea4c344828aa8ab0392f1b71c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:41.559115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:41.647019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:41.648434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5b1dbdea4c344828aa8ab0392f1b71c', 'request_id': 'req-b792694a-067f-4b71-86d5-a500f920ef7b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.659687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5b1dbdea4c344828aa8ab0392f1b71c', 'request_id': 'req-b792694a-067f-4b71-86d5-a500f920ef7b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.667993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5b1dbdea4c344828aa8ab0392f1b71c', 'request_id': 'req-b792694a-067f-4b71-86d5-a500f920ef7b'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.686483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] The token cache is being invalidate because project e5b1dbdea4c344828aa8ab0392f1b71c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:41.687007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.687781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.690042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1162/2321] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:41 2022] DELETE /identity/v3/projects/e5b1dbdea4c344828aa8ab0392f1b71c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:41.694123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c11c3-160d-4831-b777-521d4648f19e service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:41.709329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c11c3-160d-4831-b777-521d4648f19e service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:41.811162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:41.812068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:41.812414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:41.812692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] PATH_INFO: `/v3/users/1a2ae69bfa224dce82887632dac03de7` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:41.818256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a2ae69bfa224dce82887632dac03de7)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:41.820350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:41.845067 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:41.909014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a2ae69bfa224dce82887632dac03de7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.934282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a2ae69bfa224dce82887632dac03de7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.955587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a2ae69bfa224dce82887632dac03de7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.956591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a2ae69bfa224dce82887632dac03de7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.957861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce85d494-8812-4d34-a27c-3b7eff7809db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a2ae69bfa224dce82887632dac03de7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:41.981722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1160/2322] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:41 2022] DELETE /identity/v3/users/1a2ae69bfa224dce82887632dac03de7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:42.407099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:42.409280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:42.442642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792694a-067f-4b71-86d5-a500f920ef7b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:42.567695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:42.574379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:42.575774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:42.576293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] PATH_INFO: `/v3/projects/c63588e3ee834ab29341df90281ee2a1` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:42.578089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c63588e3ee834ab29341df90281ee2a1)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:42.580516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:42.661363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c63588e3ee834ab29341df90281ee2a1', 'request_id': 'req-a6a9d540-fc65-453d-89d9-ec79c8a07d11'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:42.674095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c63588e3ee834ab29341df90281ee2a1', 'request_id': 'req-a6a9d540-fc65-453d-89d9-ec79c8a07d11'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:42.684637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c63588e3ee834ab29341df90281ee2a1', 'request_id': 'req-a6a9d540-fc65-453d-89d9-ec79c8a07d11'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:42.698091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] The token cache is being invalidate because project c63588e3ee834ab29341df90281ee2a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:42.698376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:42.699055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a6a9d540-fc65-453d-89d9-ec79c8a07d11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:42.704543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1163/2323] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:42 2022] DELETE /identity/v3/projects/c63588e3ee834ab29341df90281ee2a1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:42.708767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0585c2-266c-4824-ba28-52a828fb61c3 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:42.709813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0585c2-266c-4824-ba28-52a828fb61c3 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:42.710344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0585c2-266c-4824-ba28-52a828fb61c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:42.733033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1d0585c2-266c-4824-ba28-52a828fb61c3 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:42.734674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1d0585c2-266c-4824-ba28-52a828fb61c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:42.851974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1161/2324] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:42.863999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660adca-2069-4d52-ac33-4af987b58d61 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:42.864332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660adca-2069-4d52-ac33-4af987b58d61 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:42.864598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4660adca-2069-4d52-ac33-4af987b58d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:42.886648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4660adca-2069-4d52-ac33-4af987b58d61 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:42.887885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4660adca-2069-4d52-ac33-4af987b58d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:42.899393 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1164/2325] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:42.903798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0585c2-266c-4824-ba28-52a828fb61c3 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:42.918080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0585c2-266c-4824-ba28-52a828fb61c3 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:42.923973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:42.924816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:42.925197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:42.925493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:42.926330 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:25:42.926870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:42.928541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:42.945100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1162/2326] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:25:42.950243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660adca-2069-4d52-ac33-4af987b58d61 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:42.961164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4660adca-2069-4d52-ac33-4af987b58d61 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:42.965970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f095e1-d2a7-42ca-8df7-92e1dd17c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:42.966643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f095e1-d2a7-42ca-8df7-92e1dd17c130 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:42.966926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f095e1-d2a7-42ca-8df7-92e1dd17c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:42.967208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f095e1-d2a7-42ca-8df7-92e1dd17c130 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:42.967905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f095e1-d2a7-42ca-8df7-92e1dd17c130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:42.970141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f095e1-d2a7-42ca-8df7-92e1dd17c130 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:43.004830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1165/2327] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:42 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:43.008501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:43.020246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4b140-3f0a-46b9-a081-74280721cf87 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:43.083697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232267b-0fff-4fcc-abaf-130bf041b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:43.084690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232267b-0fff-4fcc-abaf-130bf041b830 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:43.085080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232267b-0fff-4fcc-abaf-130bf041b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:43.085429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232267b-0fff-4fcc-abaf-130bf041b830 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:43.086252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232267b-0fff-4fcc-abaf-130bf041b830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:43.088395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0232267b-0fff-4fcc-abaf-130bf041b830 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:43.130200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f095e1-d2a7-42ca-8df7-92e1dd17c130 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:43.131825 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1163/2328] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:43 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:43.142498 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f095e1-d2a7-42ca-8df7-92e1dd17c130 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:43.160038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16402d-8341-4b5c-8514-cbe9c57ab718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:43.160976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16402d-8341-4b5c-8514-cbe9c57ab718 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:43.161329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16402d-8341-4b5c-8514-cbe9c57ab718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:43.161699 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16402d-8341-4b5c-8514-cbe9c57ab718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:43.164120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16402d-8341-4b5c-8514-cbe9c57ab718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:43.165965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16402d-8341-4b5c-8514-cbe9c57ab718 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:43.196001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1166/2329] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:25:43.200608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232267b-0fff-4fcc-abaf-130bf041b830 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:43.238472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232267b-0fff-4fcc-abaf-130bf041b830 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:43.274913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411257a-819b-47ee-bba5-1a135f138f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:43.282810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411257a-819b-47ee-bba5-1a135f138f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:43.283112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411257a-819b-47ee-bba5-1a135f138f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:43.283544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411257a-819b-47ee-bba5-1a135f138f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:43.284459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411257a-819b-47ee-bba5-1a135f138f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:43.286736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411257a-819b-47ee-bba5-1a135f138f0a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:43.295570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:43.329491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1164/2330] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:43.330871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16402d-8341-4b5c-8514-cbe9c57ab718 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:43.345047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16402d-8341-4b5c-8514-cbe9c57ab718 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:43.363361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b068e-6daa-4a08-af37-e7f82b17261f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:43.364568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b068e-6daa-4a08-af37-e7f82b17261f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:43.365016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b068e-6daa-4a08-af37-e7f82b17261f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:43.365443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b068e-6daa-4a08-af37-e7f82b17261f admin admin] PATH_INFO: `/v3/projects/12feba9533014c0abb9f79d707c34da0/users/b4319376ec8e4240867819160c44a0bf/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:43.385609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b068e-6daa-4a08-af37-e7f82b17261f admin admin] RBAC: Authorizing `identity:create_grant(project_id=12feba9533014c0abb9f79d707c34da0, user_id=b4319376ec8e4240867819160c44a0bf, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:43.388959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b068e-6daa-4a08-af37-e7f82b17261f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:43.414188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1167/2331] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:43 2022] PUT /identity/v3/projects/12feba9533014c0abb9f79d707c34da0/users/b4319376ec8e4240867819160c44a0bf/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:43.429490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739804da-076a-4f5a-8fda-1b7be3797300 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:43.431510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739804da-076a-4f5a-8fda-1b7be3797300 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:43.431510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739804da-076a-4f5a-8fda-1b7be3797300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:43.472344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-739804da-076a-4f5a-8fda-1b7be3797300 None None] MFA Rules not processed for user `b4319376ec8e4240867819160c44a0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:43.473922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-739804da-076a-4f5a-8fda-1b7be3797300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:43.587035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1165/2332] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:43.589090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b068e-6daa-4a08-af37-e7f82b17261f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:43.594139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:43.719186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b068e-6daa-4a08-af37-e7f82b17261f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:43.730348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8b1a8-55eb-4f0f-bf6f-f3a305cee44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:43.731186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b1a8-55eb-4f0f-bf6f-f3a305cee44e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:43.731575 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b1a8-55eb-4f0f-bf6f-f3a305cee44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:43.732103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8b1a8-55eb-4f0f-bf6f-f3a305cee44e admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:43.732913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8b1a8-55eb-4f0f-bf6f-f3a305cee44e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:43.742426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8b1a8-55eb-4f0f-bf6f-f3a305cee44e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:43.800876 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1168/2333] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:43 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:43.802084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739804da-076a-4f5a-8fda-1b7be3797300 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:43.817553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739804da-076a-4f5a-8fda-1b7be3797300 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:43.918051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad71cb-13c0-4896-b50f-3f78228b57c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:43.919845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad71cb-13c0-4896-b50f-3f78228b57c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:43.920369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad71cb-13c0-4896-b50f-3f78228b57c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:43.920878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad71cb-13c0-4896-b50f-3f78228b57c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:43.922796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad71cb-13c0-4896-b50f-3f78228b57c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:43.926073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad71cb-13c0-4896-b50f-3f78228b57c3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:43.970163 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1166/2334] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:43 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:25:43.971411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8b1a8-55eb-4f0f-bf6f-f3a305cee44e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:43.988810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8b1a8-55eb-4f0f-bf6f-f3a305cee44e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:43.996730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8783b1-6d7f-48fd-b3a5-dd618b38985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:43.997809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8783b1-6d7f-48fd-b3a5-dd618b38985c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:43.998250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8783b1-6d7f-48fd-b3a5-dd618b38985c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:43.998700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8783b1-6d7f-48fd-b3a5-dd618b38985c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:44.000410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8783b1-6d7f-48fd-b3a5-dd618b38985c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:44.003197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8783b1-6d7f-48fd-b3a5-dd618b38985c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:44.022954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1169/2335] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:44.026885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad71cb-13c0-4896-b50f-3f78228b57c3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:44.042792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad71cb-13c0-4896-b50f-3f78228b57c3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:44.049327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa81c98-87a3-4b5e-8219-a80910cba05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:44.050191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa81c98-87a3-4b5e-8219-a80910cba05e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:44.050572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa81c98-87a3-4b5e-8219-a80910cba05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:44.050948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa81c98-87a3-4b5e-8219-a80910cba05e admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:44.052065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa81c98-87a3-4b5e-8219-a80910cba05e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:44.053832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa81c98-87a3-4b5e-8219-a80910cba05e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:44.070584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1167/2336] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:25:44.072105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8783b1-6d7f-48fd-b3a5-dd618b38985c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:44.083442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8783b1-6d7f-48fd-b3a5-dd618b38985c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:44.089834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f2469-bbfb-409b-8a9c-f35afe391f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:44.090898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f2469-bbfb-409b-8a9c-f35afe391f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:44.091271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f2469-bbfb-409b-8a9c-f35afe391f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:44.091744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f2469-bbfb-409b-8a9c-f35afe391f0e admin admin] PATH_INFO: `/v3/projects/353833c3039545a9b842d610607251fd/users/d59c6b581e164a2987117de562275b89/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:44.120191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f2469-bbfb-409b-8a9c-f35afe391f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=353833c3039545a9b842d610607251fd, user_id=d59c6b581e164a2987117de562275b89, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:44.123086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f2469-bbfb-409b-8a9c-f35afe391f0e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:44.149748 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1170/2337] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:44 2022] PUT /identity/v3/projects/353833c3039545a9b842d610607251fd/users/d59c6b581e164a2987117de562275b89/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:44.156727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb143b4f-723f-4001-89fa-cd0f001c7017 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:44.157199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb143b4f-723f-4001-89fa-cd0f001c7017 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:44.157479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb143b4f-723f-4001-89fa-cd0f001c7017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:44.195213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-eb143b4f-723f-4001-89fa-cd0f001c7017 None None] MFA Rules not processed for user `d59c6b581e164a2987117de562275b89`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:44.196572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-eb143b4f-723f-4001-89fa-cd0f001c7017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:44.314157 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:44.334158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1168/2338] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:44.335500 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f2469-bbfb-409b-8a9c-f35afe391f0e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:44.509263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f2469-bbfb-409b-8a9c-f35afe391f0e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:44.520499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdaa01-d21c-46cd-8976-31b642693911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:44.521224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdaa01-d21c-46cd-8976-31b642693911 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:44.521508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdaa01-d21c-46cd-8976-31b642693911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:44.522655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdaa01-d21c-46cd-8976-31b642693911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:44.524433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdaa01-d21c-46cd-8976-31b642693911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:44.530110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdaa01-d21c-46cd-8976-31b642693911 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:44.583347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1171/2339] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:44 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:44.599470 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb143b4f-723f-4001-89fa-cd0f001c7017 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:44.627775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb143b4f-723f-4001-89fa-cd0f001c7017 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:44.800821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eabf7d-8a04-4fd1-a700-0a169935f5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:44.802194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eabf7d-8a04-4fd1-a700-0a169935f5dc admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:44.802764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eabf7d-8a04-4fd1-a700-0a169935f5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:44.802764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eabf7d-8a04-4fd1-a700-0a169935f5dc admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:44.803456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eabf7d-8a04-4fd1-a700-0a169935f5dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:44.809081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eabf7d-8a04-4fd1-a700-0a169935f5dc admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:44.853889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1169/2340] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:44 2022] POST /identity/v3/users => generated 299 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:44.858683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdaa01-d21c-46cd-8976-31b642693911 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:44.876912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdaa01-d21c-46cd-8976-31b642693911 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:44.886408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e5fb8-227d-4453-8f12-3e00a607adb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:44.890627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5fb8-227d-4453-8f12-3e00a607adb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:44.890627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5fb8-227d-4453-8f12-3e00a607adb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:44.891314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e5fb8-227d-4453-8f12-3e00a607adb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:44.892474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e5fb8-227d-4453-8f12-3e00a607adb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:44.894542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e5fb8-227d-4453-8f12-3e00a607adb1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:44.919589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1172/2341] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:44.925373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eabf7d-8a04-4fd1-a700-0a169935f5dc admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:44.952226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eabf7d-8a04-4fd1-a700-0a169935f5dc admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:44.957961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f8438-a2a5-4d2f-abb7-f9d105e8fa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:44.958865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f8438-a2a5-4d2f-abb7-f9d105e8fa3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:44.959144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f8438-a2a5-4d2f-abb7-f9d105e8fa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:44.959411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f8438-a2a5-4d2f-abb7-f9d105e8fa3c admin admin] PATH_INFO: `/v3/projects/9986700228fb47f988e0878c077cda25/users/2ef986b58c7f4151b20162c36e6f8654/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:44.979209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f8438-a2a5-4d2f-abb7-f9d105e8fa3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9986700228fb47f988e0878c077cda25, user_id=2ef986b58c7f4151b20162c36e6f8654, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:44.982256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f8438-a2a5-4d2f-abb7-f9d105e8fa3c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:45.042788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1170/2342] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:44 2022] PUT /identity/v3/projects/9986700228fb47f988e0878c077cda25/users/2ef986b58c7f4151b20162c36e6f8654/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:45.047477 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e5fb8-227d-4453-8f12-3e00a607adb1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:45.059035 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e5fb8-227d-4453-8f12-3e00a607adb1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:45.096296 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:45.138063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d68f7-404d-4ee2-af95-26205fc7bf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:45.139499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d68f7-404d-4ee2-af95-26205fc7bf75 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:45.139499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d68f7-404d-4ee2-af95-26205fc7bf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:45.139499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d68f7-404d-4ee2-af95-26205fc7bf75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:45.140294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d68f7-404d-4ee2-af95-26205fc7bf75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:45.142001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707d68f7-404d-4ee2-af95-26205fc7bf75 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:45.154138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1173/2343] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:45.158429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f8438-a2a5-4d2f-abb7-f9d105e8fa3c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:45.170290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f8438-a2a5-4d2f-abb7-f9d105e8fa3c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:45.178368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:45.179286 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:45.179619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:45.179831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] PATH_INFO: `/v3/projects/9986700228fb47f988e0878c077cda25/users/2ef986b58c7f4151b20162c36e6f8654/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:45.181958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9986700228fb47f988e0878c077cda25, user_id=2ef986b58c7f4151b20162c36e6f8654, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:45.184657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:45.197303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1171/2344] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:45 2022] PUT /identity/v3/projects/9986700228fb47f988e0878c077cda25/users/2ef986b58c7f4151b20162c36e6f8654/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:45.209219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ce1c-b892-4451-8a76-a847de9c50c8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:45.209219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ce1c-b892-4451-8a76-a847de9c50c8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:45.210012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230ce1c-b892-4451-8a76-a847de9c50c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:45.253271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3230ce1c-b892-4451-8a76-a847de9c50c8 None None] MFA Rules not processed for user `2ef986b58c7f4151b20162c36e6f8654`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:45.262536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3230ce1c-b892-4451-8a76-a847de9c50c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:45.425323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1174/2345] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:45.436034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:45.437832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:45.548439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b303b0-c634-4b4c-b2fa-80493d9672ca admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:45.553605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159214fc-a846-4233-a8f0-e6232b980079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:45.554337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159214fc-a846-4233-a8f0-e6232b980079 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:45.554849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159214fc-a846-4233-a8f0-e6232b980079 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:45.555225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159214fc-a846-4233-a8f0-e6232b980079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:45.592457 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159214fc-a846-4233-a8f0-e6232b980079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:45.594299 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159214fc-a846-4233-a8f0-e6232b980079 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:45.634855 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1172/2346] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:46.741538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230ce1c-b892-4451-8a76-a847de9c50c8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:46.743994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3230ce1c-b892-4451-8a76-a847de9c50c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:46.760177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230ce1c-b892-4451-8a76-a847de9c50c8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:46.764792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229f0d7-9164-4061-8f9e-0a98c7bf5797 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:46.766171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229f0d7-9164-4061-8f9e-0a98c7bf5797 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:46.766582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229f0d7-9164-4061-8f9e-0a98c7bf5797 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:46.767016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229f0d7-9164-4061-8f9e-0a98c7bf5797 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:46.855516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229f0d7-9164-4061-8f9e-0a98c7bf5797 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:46.857201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229f0d7-9164-4061-8f9e-0a98c7bf5797 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:46.879990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1175/2347] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:51.329618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159214fc-a846-4233-a8f0-e6232b980079 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:51.345535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159214fc-a846-4233-a8f0-e6232b980079 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:51.356642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add673bf-d7a8-470f-8179-0ee2fae697f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:51.358326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add673bf-d7a8-470f-8179-0ee2fae697f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:51.358775 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add673bf-d7a8-470f-8179-0ee2fae697f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:51.359172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add673bf-d7a8-470f-8179-0ee2fae697f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:51.433076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:51.450295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add673bf-d7a8-470f-8179-0ee2fae697f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:51.451997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add673bf-d7a8-470f-8179-0ee2fae697f1 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:51.465906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1173/2348] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:53.924178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229f0d7-9164-4061-8f9e-0a98c7bf5797 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:53.986636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:54.002659 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229f0d7-9164-4061-8f9e-0a98c7bf5797 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:54.007424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:54.008320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:54.008602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:54.008876 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] PATH_INFO: `/v3/users/b4319376ec8e4240867819160c44a0bf` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:54.010134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4319376ec8e4240867819160c44a0bf)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:54.012090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:54.068072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4319376ec8e4240867819160c44a0bf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.076641 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4319376ec8e4240867819160c44a0bf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.088888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4319376ec8e4240867819160c44a0bf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.089609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4319376ec8e4240867819160c44a0bf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.090248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4319376ec8e4240867819160c44a0bf'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.104492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1176/2349] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:53 2022] DELETE /identity/v3/users/b4319376ec8e4240867819160c44a0bf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:54.110111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde1c42-63c0-4f1c-afac-8714607184de None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:54.111128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde1c42-63c0-4f1c-afac-8714607184de None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:54.111128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde1c42-63c0-4f1c-afac-8714607184de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:54.129804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0cde1c42-63c0-4f1c-afac-8714607184de None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:54.131113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0cde1c42-63c0-4f1c-afac-8714607184de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:54.203684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1174/2350] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:54.214231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:54.215814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:54.291565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d92fb-4702-4c58-8217-8ac95379d1b0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:54.295954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4443a-7c78-47c9-9e14-ae8e357058de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:54.296737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4443a-7c78-47c9-9e14-ae8e357058de service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:54.297390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4443a-7c78-47c9-9e14-ae8e357058de service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:54.297390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4443a-7c78-47c9-9e14-ae8e357058de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:54.308599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4443a-7c78-47c9-9e14-ae8e357058de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:54.310146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4443a-7c78-47c9-9e14-ae8e357058de service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:54.321417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1177/2351] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:54.647020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde1c42-63c0-4f1c-afac-8714607184de None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:54.648805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0cde1c42-63c0-4f1c-afac-8714607184de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:54.664449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde1c42-63c0-4f1c-afac-8714607184de None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:54.670371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:54.671527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:54.671683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:54.672268 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] PATH_INFO: `/v3/projects/12feba9533014c0abb9f79d707c34da0` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:54.673097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12feba9533014c0abb9f79d707c34da0)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:54.675079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:54.754446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12feba9533014c0abb9f79d707c34da0', 'request_id': 'req-1815f0b2-795b-4dbd-b467-dc664d7d86d4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.761417 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12feba9533014c0abb9f79d707c34da0', 'request_id': 'req-1815f0b2-795b-4dbd-b467-dc664d7d86d4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.775544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12feba9533014c0abb9f79d707c34da0', 'request_id': 'req-1815f0b2-795b-4dbd-b467-dc664d7d86d4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.789906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] The token cache is being invalidate because project 12feba9533014c0abb9f79d707c34da0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:54.790209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.790996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.792868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1175/2352] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:54 2022] DELETE /identity/v3/projects/12feba9533014c0abb9f79d707c34da0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:54.797372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4443a-7c78-47c9-9e14-ae8e357058de service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:54.808470 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4443a-7c78-47c9-9e14-ae8e357058de service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:54.871283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:54.872226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:54.872520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:54.872799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] PATH_INFO: `/v3/users/d59c6b581e164a2987117de562275b89` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:54.874070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d59c6b581e164a2987117de562275b89)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:54.876104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:54.887840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:54.930858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd59c6b581e164a2987117de562275b89'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.939333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd59c6b581e164a2987117de562275b89'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.952417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd59c6b581e164a2987117de562275b89'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.953201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd59c6b581e164a2987117de562275b89'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.953924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd59c6b581e164a2987117de562275b89'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:54.968071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1178/2353] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:54 2022] DELETE /identity/v3/users/d59c6b581e164a2987117de562275b89 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:55.279815 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:55.281716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:55.306866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815f0b2-795b-4dbd-b467-dc664d7d86d4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:55.373252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:55.374093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:55.374758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:55.375043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] PATH_INFO: `/v3/projects/353833c3039545a9b842d610607251fd` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:55.376258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=353833c3039545a9b842d610607251fd)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:55.378407 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:55.395577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:55.430091 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353833c3039545a9b842d610607251fd', 'request_id': 'req-988a6d31-a893-4b3e-a588-fa73b36a1484'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.437272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353833c3039545a9b842d610607251fd', 'request_id': 'req-988a6d31-a893-4b3e-a588-fa73b36a1484'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.450151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353833c3039545a9b842d610607251fd', 'request_id': 'req-988a6d31-a893-4b3e-a588-fa73b36a1484'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.461260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] The token cache is being invalidate because project 353833c3039545a9b842d610607251fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:55.461553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.462249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.463813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1176/2354] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:55 2022] DELETE /identity/v3/projects/353833c3039545a9b842d610607251fd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:55.467809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:55.469760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:55.484131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e998c1-a820-4a58-a48b-d9e4e1f68f31 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:55.548996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:55.549966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:55.550219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:55.550496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] PATH_INFO: `/v3/users/2ef986b58c7f4151b20162c36e6f8654` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:55.551741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef986b58c7f4151b20162c36e6f8654)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:55.553838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:55.612647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef986b58c7f4151b20162c36e6f8654'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.623727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef986b58c7f4151b20162c36e6f8654'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.648708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef986b58c7f4151b20162c36e6f8654'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.649592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef986b58c7f4151b20162c36e6f8654'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.650376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e39b2bcd-1f21-48e2-9812-bb0d075f3ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef986b58c7f4151b20162c36e6f8654'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:55.677048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1179/2355] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:25:55 2022] DELETE /identity/v3/users/2ef986b58c7f4151b20162c36e6f8654 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:56.090973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:56.095116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:56.108351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a6d31-a893-4b3e-a588-fa73b36a1484 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:56.174332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:56.175216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.175497 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.175776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] PATH_INFO: `/v3/projects/9986700228fb47f988e0878c077cda25` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.177162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9986700228fb47f988e0878c077cda25)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:56.179228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:56.234006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9986700228fb47f988e0878c077cda25', 'request_id': 'req-37a61256-98a3-4c48-be25-046d5b77a460'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:56.241206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9986700228fb47f988e0878c077cda25', 'request_id': 'req-37a61256-98a3-4c48-be25-046d5b77a460'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:56.252020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9986700228fb47f988e0878c077cda25', 'request_id': 'req-37a61256-98a3-4c48-be25-046d5b77a460'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:56.265057 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] The token cache is being invalidate because project 9986700228fb47f988e0878c077cda25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:25:56.265368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:56.266061 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-37a61256-98a3-4c48-be25-046d5b77a460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:25:56.267995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1177/2356] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:25:56 2022] DELETE /identity/v3/projects/9986700228fb47f988e0878c077cda25 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:25:56.274111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beceaf2-fee3-42be-83f9-7a1e7ea8f2b9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.274760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beceaf2-fee3-42be-83f9-7a1e7ea8f2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.274760 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beceaf2-fee3-42be-83f9-7a1e7ea8f2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.287571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:56.294508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0beceaf2-fee3-42be-83f9-7a1e7ea8f2b9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:56.296548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0beceaf2-fee3-42be-83f9-7a1e7ea8f2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:56.379965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1180/2357] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:25:56.388737 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0e7b8-de64-4674-8fc0-a54029948741 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.389259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0e7b8-de64-4674-8fc0-a54029948741 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.389358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0e7b8-de64-4674-8fc0-a54029948741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.408688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-54c0e7b8-de64-4674-8fc0-a54029948741 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:56.409975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-54c0e7b8-de64-4674-8fc0-a54029948741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:56.421989 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1178/2358] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:56.423517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beceaf2-fee3-42be-83f9-7a1e7ea8f2b9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:56.435000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beceaf2-fee3-42be-83f9-7a1e7ea8f2b9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:56.448219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:56.457904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.458203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.458487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.459290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:25:56.459606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:56.461287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:56.492431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1181/2359] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:25:56.496950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0e7b8-de64-4674-8fc0-a54029948741 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:56.510317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0e7b8-de64-4674-8fc0-a54029948741 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:56.515501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714ed6f-307d-4357-843f-f4a40aeaf0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:56.516397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714ed6f-307d-4357-843f-f4a40aeaf0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.516857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714ed6f-307d-4357-843f-f4a40aeaf0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.517134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714ed6f-307d-4357-843f-f4a40aeaf0e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.517883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714ed6f-307d-4357-843f-f4a40aeaf0e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:56.519903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714ed6f-307d-4357-843f-f4a40aeaf0e3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:56.554838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1179/2360] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:25:56 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:25:56.558696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:56.570674 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ae0e9-d239-4e1c-969e-fa05cd39e3b2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:56.628245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f0508-c1e2-467e-b179-0a236389d017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:56.628942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0508-c1e2-467e-b179-0a236389d017 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.629226 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0508-c1e2-467e-b179-0a236389d017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.629504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0508-c1e2-467e-b179-0a236389d017 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.630251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f0508-c1e2-467e-b179-0a236389d017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:56.632225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f0508-c1e2-467e-b179-0a236389d017 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:56.655070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1182/2361] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:25:56 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:25:56.659175 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714ed6f-307d-4357-843f-f4a40aeaf0e3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:56.666205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:56.670136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714ed6f-307d-4357-843f-f4a40aeaf0e3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:56.677010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba3c88-617c-4a5d-b3dd-1eed951f8b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:56.677981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba3c88-617c-4a5d-b3dd-1eed951f8b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.678392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba3c88-617c-4a5d-b3dd-1eed951f8b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.679097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba3c88-617c-4a5d-b3dd-1eed951f8b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.679873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba3c88-617c-4a5d-b3dd-1eed951f8b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:56.681984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ba3c88-617c-4a5d-b3dd-1eed951f8b8e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:56.693381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1180/2362] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:56.697601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f0508-c1e2-467e-b179-0a236389d017 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:56.709992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f0508-c1e2-467e-b179-0a236389d017 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:56.714893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f99fdc-74e9-4c6e-8ff3-d210096b73bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:56.715574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f99fdc-74e9-4c6e-8ff3-d210096b73bd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.715861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f99fdc-74e9-4c6e-8ff3-d210096b73bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.716138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f99fdc-74e9-4c6e-8ff3-d210096b73bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.716918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f99fdc-74e9-4c6e-8ff3-d210096b73bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:56.719000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f99fdc-74e9-4c6e-8ff3-d210096b73bd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:56.730068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1183/2363] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:25:56.734656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba3c88-617c-4a5d-b3dd-1eed951f8b8e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:56.746019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ba3c88-617c-4a5d-b3dd-1eed951f8b8e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:56.753098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:56.753098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.753098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.753098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] PATH_INFO: `/v3/projects/abdc518ee18645c184f11d3c9267677c/users/2842003f3cf5441fbdb1cdcffe69c6d7/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.771314 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=abdc518ee18645c184f11d3c9267677c, user_id=2842003f3cf5441fbdb1cdcffe69c6d7, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:56.774929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:56.797762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1181/2364] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:25:56 2022] PUT /identity/v3/projects/abdc518ee18645c184f11d3c9267677c/users/2842003f3cf5441fbdb1cdcffe69c6d7/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:25:56.808293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b36cb0-4036-42d7-b5d2-65e78bb12cfb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:56.808552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b36cb0-4036-42d7-b5d2-65e78bb12cfb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:56.808916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b36cb0-4036-42d7-b5d2-65e78bb12cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:56.856461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-50b36cb0-4036-42d7-b5d2-65e78bb12cfb None None] MFA Rules not processed for user `2842003f3cf5441fbdb1cdcffe69c6d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:56.857841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-50b36cb0-4036-42d7-b5d2-65e78bb12cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:56.990508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1184/2365] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:25:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:57.011368 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:57.013537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:57.105029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62608d2-b6b3-460e-aef2-55aad8118fde admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:57.110934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1e761-b5b1-405d-b4a8-205e8e173f89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:57.111935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1e761-b5b1-405d-b4a8-205e8e173f89 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:57.112411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1e761-b5b1-405d-b4a8-205e8e173f89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:57.113132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1e761-b5b1-405d-b4a8-205e8e173f89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:57.141306 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1e761-b5b1-405d-b4a8-205e8e173f89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:57.142857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1e761-b5b1-405d-b4a8-205e8e173f89 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:57.153885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1182/2366] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:25:58.199586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e818d33-1cd1-4827-abcc-ec5cf81749cd None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.200167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e818d33-1cd1-4827-abcc-ec5cf81749cd None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.200285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e818d33-1cd1-4827-abcc-ec5cf81749cd None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.202014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1185/2367] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:25:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:25:58.209458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a64b8-c7f0-4f79-b060-e6958a00bc27 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.210001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a64b8-c7f0-4f79-b060-e6958a00bc27 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.210180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a64b8-c7f0-4f79-b060-e6958a00bc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.229414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3d5a64b8-c7f0-4f79-b060-e6958a00bc27 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:58.230883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3d5a64b8-c7f0-4f79-b060-e6958a00bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:58.303078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1183/2368] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:25:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:58.315892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e818d33-1cd1-4827-abcc-ec5cf81749cd None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:58.317708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1e818d33-1cd1-4827-abcc-ec5cf81749cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:58.323569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:25:58.385704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e818d33-1cd1-4827-abcc-ec5cf81749cd None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:58.389750 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646c01-4e96-4a2e-9f47-ecb566fc11b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:58.390511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646c01-4e96-4a2e-9f47-ecb566fc11b7 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.390802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646c01-4e96-4a2e-9f47-ecb566fc11b7 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.391086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646c01-4e96-4a2e-9f47-ecb566fc11b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.403078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646c01-4e96-4a2e-9f47-ecb566fc11b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:58.404543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646c01-4e96-4a2e-9f47-ecb566fc11b7 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:58.417471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1186/2369] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:58.490548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0e4c-8969-4f72-b455-3dce5dcf2488 None None] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.490838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0e4c-8969-4f72-b455-3dce5dcf2488 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.491156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0e4c-8969-4f72-b455-3dce5dcf2488 None None] PATH_INFO: `/` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.492559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1184/2370] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 18 14:25:58.497571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8c4f9-649f-4442-9371-1898c78f6656 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.497917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8c4f9-649f-4442-9371-1898c78f6656 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.498203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8c4f9-649f-4442-9371-1898c78f6656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.521350 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-aeb8c4f9-649f-4442-9371-1898c78f6656 None None] MFA Rules not processed for user `1f1fe084c3164e50bf5d904350925f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:58.522612 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-aeb8c4f9-649f-4442-9371-1898c78f6656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:58.581911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1187/2371] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:25:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:58.596519 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0e4c-8969-4f72-b455-3dce5dcf2488 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:58.598404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-de6e0e4c-8969-4f72-b455-3dce5dcf2488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:58.612665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0e4c-8969-4f72-b455-3dce5dcf2488 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:58.616559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cc63f-65ec-4c58-a658-506bd32269cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:58.617206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cc63f-65ec-4c58-a658-506bd32269cf service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.617486 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cc63f-65ec-4c58-a658-506bd32269cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.617828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cc63f-65ec-4c58-a658-506bd32269cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.634356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cc63f-65ec-4c58-a658-506bd32269cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:58.634356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cc63f-65ec-4c58-a658-506bd32269cf service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:58.636386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1185/2372] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:25:58.731534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550517-ca34-409d-8a8e-eb01ea9ae4fa None None] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.732224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550517-ca34-409d-8a8e-eb01ea9ae4fa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.732596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6550517-ca34-409d-8a8e-eb01ea9ae4fa None None] PATH_INFO: `/` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.734147 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1188/2373] 10.222.0.71 () {54 vars in 921 bytes} [Mon Jul 18 14:25:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 18 14:25:58.742083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9528d-2dca-4af9-862d-de7db37c2479 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.742318 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9528d-2dca-4af9-862d-de7db37c2479 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.742772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9528d-2dca-4af9-862d-de7db37c2479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.762458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-55f9528d-2dca-4af9-862d-de7db37c2479 None None] MFA Rules not processed for user `4fffb91636744fc39886bdf568317ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:25:58.763679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-55f9528d-2dca-4af9-862d-de7db37c2479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:25:58.773694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1186/2374] 10.222.0.71 () {58 vars in 1018 bytes} [Mon Jul 18 14:25:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:25:58.786649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6550517-ca34-409d-8a8e-eb01ea9ae4fa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:25:58.799826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6550517-ca34-409d-8a8e-eb01ea9ae4fa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:25:58.808758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:25:58.809546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:25:58.810000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:25:58.810373 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:25:58.826991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:25:58.828620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:25:58.840433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1189/2375] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:25:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:03.223514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540771b-d559-43ee-a3d3-02d2eb6b3d32 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:03.224001 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540771b-d559-43ee-a3d3-02d2eb6b3d32 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:03.224104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540771b-d559-43ee-a3d3-02d2eb6b3d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:03.244214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f540771b-d559-43ee-a3d3-02d2eb6b3d32 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:03.245483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f540771b-d559-43ee-a3d3-02d2eb6b3d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:03.289965 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:03.307585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1187/2376] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:26:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:03.320724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:03.322785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:03.414003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b23f7f-876d-41c5-aeda-650d5cf30556 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:03.422202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf658f09-8ad1-4510-994e-98bfb05ac79f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:03.422837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf658f09-8ad1-4510-994e-98bfb05ac79f service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:03.423123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf658f09-8ad1-4510-994e-98bfb05ac79f service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:03.423401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf658f09-8ad1-4510-994e-98bfb05ac79f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:03.432868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf658f09-8ad1-4510-994e-98bfb05ac79f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:03.434525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf658f09-8ad1-4510-994e-98bfb05ac79f service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:03.443756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1190/2377] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:26:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:26:13.300452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:13.439969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:18.290982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540771b-d559-43ee-a3d3-02d2eb6b3d32 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.358723 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540771b-d559-43ee-a3d3-02d2eb6b3d32 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:18.364593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:18.365483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.365802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.366077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] PATH_INFO: `/v3/users/2842003f3cf5441fbdb1cdcffe69c6d7` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.367400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2842003f3cf5441fbdb1cdcffe69c6d7)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:18.369378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:18.419921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2842003f3cf5441fbdb1cdcffe69c6d7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.428284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2842003f3cf5441fbdb1cdcffe69c6d7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.441166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2842003f3cf5441fbdb1cdcffe69c6d7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.441947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2842003f3cf5441fbdb1cdcffe69c6d7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.442584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2842003f3cf5441fbdb1cdcffe69c6d7'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:18.456271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1188/2378] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:26:18 2022] DELETE /identity/v3/users/2842003f3cf5441fbdb1cdcffe69c6d7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:18.461001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9db5f-8195-4f2d-a32d-0237421643d2 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.461596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9db5f-8195-4f2d-a32d-0237421643d2 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.461596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9db5f-8195-4f2d-a32d-0237421643d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.480881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-70c9db5f-8195-4f2d-a32d-0237421643d2 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:18.482190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-70c9db5f-8195-4f2d-a32d-0237421643d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:18.546361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1191/2379] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:26:18.557636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.559120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:18.622831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03ddc2-756d-4457-9b68-7db9800a54e9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:18.627012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bf362-292c-472b-a9dc-47fc404d9a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:18.627674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf362-292c-472b-a9dc-47fc404d9a51 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.627962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf362-292c-472b-a9dc-47fc404d9a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.628224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bf362-292c-472b-a9dc-47fc404d9a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.638707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bf362-292c-472b-a9dc-47fc404d9a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:18.640242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bf362-292c-472b-a9dc-47fc404d9a51 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:18.651084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1189/2380] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:18.946709 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9db5f-8195-4f2d-a32d-0237421643d2 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:18.948239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-70c9db5f-8195-4f2d-a32d-0237421643d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:18.961798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9db5f-8195-4f2d-a32d-0237421643d2 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:18.968327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:18.969403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:18.969819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:18.970176 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] PATH_INFO: `/v3/projects/abdc518ee18645c184f11d3c9267677c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:18.971644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=abdc518ee18645c184f11d3c9267677c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:18.973804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.028662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abdc518ee18645c184f11d3c9267677c', 'request_id': 'req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:19.037397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abdc518ee18645c184f11d3c9267677c', 'request_id': 'req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:19.045043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abdc518ee18645c184f11d3c9267677c', 'request_id': 'req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:19.058137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] The token cache is being invalidate because project abdc518ee18645c184f11d3c9267677c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:26:19.058431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:19.059249 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dd0f4ae3-57a3-4ad0-97fe-d01e02b75bfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:19.061051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1192/2381] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:26:18 2022] DELETE /identity/v3/projects/abdc518ee18645c184f11d3c9267677c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:19.067034 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436ef3-e875-4844-b44b-3c00cd63a553 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.067510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436ef3-e875-4844-b44b-3c00cd63a553 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.067792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436ef3-e875-4844-b44b-3c00cd63a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.087886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-06436ef3-e875-4844-b44b-3c00cd63a553 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:19.089127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-06436ef3-e875-4844-b44b-3c00cd63a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:19.108234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:19.159700 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1190/2382] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:19.167891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9ddb-926f-448c-8dc4-f42b9cd08b5d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.168177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9ddb-926f-448c-8dc4-f42b9cd08b5d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.168457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f9ddb-926f-448c-8dc4-f42b9cd08b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.187947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-144f9ddb-926f-448c-8dc4-f42b9cd08b5d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:19.189179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-144f9ddb-926f-448c-8dc4-f42b9cd08b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:19.197807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1193/2383] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:19.202216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436ef3-e875-4844-b44b-3c00cd63a553 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.214552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436ef3-e875-4844-b44b-3c00cd63a553 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.221316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.222219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.222483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.222762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.223503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:26:19.223814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.225649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.239770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1191/2384] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:26:19.243991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9ddb-926f-448c-8dc4-f42b9cd08b5d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.253692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f9ddb-926f-448c-8dc4-f42b9cd08b5d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.259001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245e87-ac21-4b8b-92e6-2f27308c9e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.259688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245e87-ac21-4b8b-92e6-2f27308c9e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.259972 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245e87-ac21-4b8b-92e6-2f27308c9e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.260236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245e87-ac21-4b8b-92e6-2f27308c9e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.261005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50245e87-ac21-4b8b-92e6-2f27308c9e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.263005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50245e87-ac21-4b8b-92e6-2f27308c9e60 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.292984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1194/2385] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:26:19 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:19.297948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.309350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92774052-edd9-4c80-b4c4-7765cc28eb73 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.379418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59d62f-8ba5-49e6-8d90-d48d5a884a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.381523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59d62f-8ba5-49e6-8d90-d48d5a884a7e admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.382028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59d62f-8ba5-49e6-8d90-d48d5a884a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.382499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59d62f-8ba5-49e6-8d90-d48d5a884a7e admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.383416 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59d62f-8ba5-49e6-8d90-d48d5a884a7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.385681 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59d62f-8ba5-49e6-8d90-d48d5a884a7e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.412200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1192/2386] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:26:19 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:19.416504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245e87-ac21-4b8b-92e6-2f27308c9e60 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.427990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245e87-ac21-4b8b-92e6-2f27308c9e60 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.433873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974cc3e-0211-4e5d-a9eb-6fdd5d4859ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.434561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cc3e-0211-4e5d-a9eb-6fdd5d4859ef admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.434838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cc3e-0211-4e5d-a9eb-6fdd5d4859ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.435106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974cc3e-0211-4e5d-a9eb-6fdd5d4859ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.435865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974cc3e-0211-4e5d-a9eb-6fdd5d4859ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.437535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974cc3e-0211-4e5d-a9eb-6fdd5d4859ef admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.448922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1195/2387] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:19.453159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59d62f-8ba5-49e6-8d90-d48d5a884a7e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.464456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59d62f-8ba5-49e6-8d90-d48d5a884a7e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.470202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9d8ff-8eea-4511-9b8c-312f284d7f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.471014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d8ff-8eea-4511-9b8c-312f284d7f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.471302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d8ff-8eea-4511-9b8c-312f284d7f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.471586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9d8ff-8eea-4511-9b8c-312f284d7f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.472998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9d8ff-8eea-4511-9b8c-312f284d7f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.474166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9d8ff-8eea-4511-9b8c-312f284d7f5a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.487686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1193/2388] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:19.492253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974cc3e-0211-4e5d-a9eb-6fdd5d4859ef admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.503085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974cc3e-0211-4e5d-a9eb-6fdd5d4859ef admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.508193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.509125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.509405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.509806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] PATH_INFO: `/v3/projects/7f49b551eeb14dbda488dbd44421faa5/users/5b16da8cf46249e38ee3705576ddc053/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.527358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f49b551eeb14dbda488dbd44421faa5, user_id=5b16da8cf46249e38ee3705576ddc053, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.530266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.544885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1196/2389] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:19 2022] PUT /identity/v3/projects/7f49b551eeb14dbda488dbd44421faa5/users/5b16da8cf46249e38ee3705576ddc053/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:19.555338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad070e-6b75-49ae-b061-5ee5c50aad37 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.555631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad070e-6b75-49ae-b061-5ee5c50aad37 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.555915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad070e-6b75-49ae-b061-5ee5c50aad37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.584540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b6ad070e-6b75-49ae-b061-5ee5c50aad37 None None] MFA Rules not processed for user `5b16da8cf46249e38ee3705576ddc053`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:19.585830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b6ad070e-6b75-49ae-b061-5ee5c50aad37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:19.661631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1194/2390] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:19.672675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:19.674694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:19.697909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:19.737719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d20d07-d82d-4b39-9f95-8849eeb13116 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:19.742197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:19.742863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:19.743141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:19.743411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:19.753603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:19.755112 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:19.764847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1197/2391] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:21.431085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db701-ad75-4b4f-9ce2-ea126b864245 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:21.431615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db701-ad75-4b4f-9ce2-ea126b864245 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:21.431615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db701-ad75-4b4f-9ce2-ea126b864245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:21.450202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-da1db701-ad75-4b4f-9ce2-ea126b864245 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:21.451436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-da1db701-ad75-4b4f-9ce2-ea126b864245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:21.466367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:21.519833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1195/2392] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 18 14:26:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:21.532666 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:21.534863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:21.619800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f9994-ed8f-407e-8cde-b84bc9108d6c service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:21.626066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d17bf-eac3-4564-8309-8ca5c7f02e21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:21.626741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d17bf-eac3-4564-8309-8ca5c7f02e21 service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:21.627056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d17bf-eac3-4564-8309-8ca5c7f02e21 service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:21.627332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d17bf-eac3-4564-8309-8ca5c7f02e21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:21.638922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d17bf-eac3-4564-8309-8ca5c7f02e21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:21.641083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d17bf-eac3-4564-8309-8ca5c7f02e21 service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:21.650559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1198/2393] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:26:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:26:22.831946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db701-ad75-4b4f-9ce2-ea126b864245 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:22.902559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db701-ad75-4b4f-9ce2-ea126b864245 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:22.907562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:22.908466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:22.908738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:22.909012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] PATH_INFO: `/v3/users/5b16da8cf46249e38ee3705576ddc053` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:22.910239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b16da8cf46249e38ee3705576ddc053)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:22.912172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:22.969047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b16da8cf46249e38ee3705576ddc053'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:22.979097 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b16da8cf46249e38ee3705576ddc053'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:22.992217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b16da8cf46249e38ee3705576ddc053'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:22.992967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b16da8cf46249e38ee3705576ddc053'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:22.993592 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b16da8cf46249e38ee3705576ddc053'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:23.006907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1196/2394] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:26:22 2022] DELETE /identity/v3/users/5b16da8cf46249e38ee3705576ddc053 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:23.011711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e4912-52d7-40eb-ab83-7066999feae7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.012223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e4912-52d7-40eb-ab83-7066999feae7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.012223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e4912-52d7-40eb-ab83-7066999feae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.030157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-773e4912-52d7-40eb-ab83-7066999feae7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:23.031390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-773e4912-52d7-40eb-ab83-7066999feae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:23.092345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1199/2395] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:23.103012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:23.105418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:23.155860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:23.170032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc17c4d-d504-4c91-a85e-125580d6d174 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:23.174240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8447e-5d6e-4c3a-ae40-b4167778bb4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:23.174894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8447e-5d6e-4c3a-ae40-b4167778bb4d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.175180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8447e-5d6e-4c3a-ae40-b4167778bb4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.175455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8447e-5d6e-4c3a-ae40-b4167778bb4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.185027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8447e-5d6e-4c3a-ae40-b4167778bb4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:23.186728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8447e-5d6e-4c3a-ae40-b4167778bb4d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:23.196438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1197/2396] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:23.498202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e4912-52d7-40eb-ab83-7066999feae7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:23.501639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-773e4912-52d7-40eb-ab83-7066999feae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:23.517974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e4912-52d7-40eb-ab83-7066999feae7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:23.524175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:23.524658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.524942 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.525246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] PATH_INFO: `/v3/projects/7f49b551eeb14dbda488dbd44421faa5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.526591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f49b551eeb14dbda488dbd44421faa5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:23.528765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:23.579210 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:23.580884 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f49b551eeb14dbda488dbd44421faa5', 'request_id': 'req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:23.590204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f49b551eeb14dbda488dbd44421faa5', 'request_id': 'req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:23.599044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f49b551eeb14dbda488dbd44421faa5', 'request_id': 'req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:23.610936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] The token cache is being invalidate because project 7f49b551eeb14dbda488dbd44421faa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:26:23.611225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:23.611968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9c81ccc4-58f0-49ac-8ad4-2e5ef1fdce99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:26:23.613787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1200/2397] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:26:23 2022] DELETE /identity/v3/projects/7f49b551eeb14dbda488dbd44421faa5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:23.619749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82555658-1e01-44fb-9eb3-1f88292b1020 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.620107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82555658-1e01-44fb-9eb3-1f88292b1020 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.620392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82555658-1e01-44fb-9eb3-1f88292b1020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.645371 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-82555658-1e01-44fb-9eb3-1f88292b1020 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:23.646753 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-82555658-1e01-44fb-9eb3-1f88292b1020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:23.708786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1198/2398] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:23.716740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d2cf-680e-41d7-b1bf-f5a8728f6748 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.717029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d2cf-680e-41d7-b1bf-f5a8728f6748 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.717877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74d2cf-680e-41d7-b1bf-f5a8728f6748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.738493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8a74d2cf-680e-41d7-b1bf-f5a8728f6748 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:23.739713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8a74d2cf-680e-41d7-b1bf-f5a8728f6748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:23.748463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1201/2399] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:23.753336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82555658-1e01-44fb-9eb3-1f88292b1020 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:23.763685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82555658-1e01-44fb-9eb3-1f88292b1020 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:23.769103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:23.769861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.770205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.770543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.771365 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:26:23.771675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:23.773308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:23.786012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1199/2400] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:26:23.790614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74d2cf-680e-41d7-b1bf-f5a8728f6748 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:23.800681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74d2cf-680e-41d7-b1bf-f5a8728f6748 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:23.806148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3be0e-20d6-4c19-8697-1b8f2328e368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:23.806809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3be0e-20d6-4c19-8697-1b8f2328e368 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.807118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3be0e-20d6-4c19-8697-1b8f2328e368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.807507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3be0e-20d6-4c19-8697-1b8f2328e368 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.808328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3be0e-20d6-4c19-8697-1b8f2328e368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:23.810308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3be0e-20d6-4c19-8697-1b8f2328e368 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:23.840527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1202/2401] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:26:23 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:23.845112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:23.855563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b081995-794c-49de-a12e-0dd136289ea5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:23.915211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad1212-d41e-4ada-a42b-b6bd93d4a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:23.915886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad1212-d41e-4ada-a42b-b6bd93d4a336 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.916174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad1212-d41e-4ada-a42b-b6bd93d4a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.916446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad1212-d41e-4ada-a42b-b6bd93d4a336 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.917136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad1212-d41e-4ada-a42b-b6bd93d4a336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:23.919226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad1212-d41e-4ada-a42b-b6bd93d4a336 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:23.941611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1200/2402] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:26:23 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:26:23.945809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3be0e-20d6-4c19-8697-1b8f2328e368 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:23.956901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3be0e-20d6-4c19-8697-1b8f2328e368 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:23.961841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441e8e-4cf3-4339-96da-08f7273d10ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:23.962444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441e8e-4cf3-4339-96da-08f7273d10ac admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.962726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441e8e-4cf3-4339-96da-08f7273d10ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.963000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de441e8e-4cf3-4339-96da-08f7273d10ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.963728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de441e8e-4cf3-4339-96da-08f7273d10ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:23.965327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de441e8e-4cf3-4339-96da-08f7273d10ac admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:23.976285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1203/2403] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:23.980373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad1212-d41e-4ada-a42b-b6bd93d4a336 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:23.991000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad1212-d41e-4ada-a42b-b6bd93d4a336 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:23.996022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae165182-ef30-4829-bed2-d83a017d49f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:23.996688 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae165182-ef30-4829-bed2-d83a017d49f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:23.996969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae165182-ef30-4829-bed2-d83a017d49f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:23.997243 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae165182-ef30-4829-bed2-d83a017d49f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:23.998009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae165182-ef30-4829-bed2-d83a017d49f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:23.999736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae165182-ef30-4829-bed2-d83a017d49f8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:24.010047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1201/2404] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:26:24.014757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441e8e-4cf3-4339-96da-08f7273d10ac admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:24.024434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de441e8e-4cf3-4339-96da-08f7273d10ac admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:24.029018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:24.030193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:24.030577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:24.030853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] PATH_INFO: `/v3/projects/3e1451a6c59a4ef0b0cfe3cc6b98c605/users/b750b0c7df7f4cee94b5b4d578f601b0/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:24.045567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1451a6c59a4ef0b0cfe3cc6b98c605, user_id=b750b0c7df7f4cee94b5b4d578f601b0, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:24.048318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:24.063195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1204/2405] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:26:24 2022] PUT /identity/v3/projects/3e1451a6c59a4ef0b0cfe3cc6b98c605/users/b750b0c7df7f4cee94b5b4d578f601b0/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:26:24.068574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad323-edee-42a9-9a90-02ffbea2963a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:24.068865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad323-edee-42a9-9a90-02ffbea2963a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:24.069144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ad323-edee-42a9-9a90-02ffbea2963a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:24.089575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-042ad323-edee-42a9-9a90-02ffbea2963a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:24.090828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-042ad323-edee-42a9-9a90-02ffbea2963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:24.147544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:24.170131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1202/2406] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:24.186163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:24.188331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:24.253828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575316a-ca39-4988-a5ad-183cddc8421f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:24.258328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:24.259008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:24.259289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:24.259566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:24.272822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:24.275167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:24.287367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1205/2407] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:24.966256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98457c7-e5e6-4fdf-8fb0-bbdee2666fe0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:24.966850 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98457c7-e5e6-4fdf-8fb0-bbdee2666fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:24.967229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98457c7-e5e6-4fdf-8fb0-bbdee2666fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:24.996666 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e98457c7-e5e6-4fdf-8fb0-bbdee2666fe0 None None] MFA Rules not processed for user `b750b0c7df7f4cee94b5b4d578f601b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:24.997966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e98457c7-e5e6-4fdf-8fb0-bbdee2666fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:25.066979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1203/2408] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:25.078196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:25.079738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:25.149809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780416a-7de7-42c9-90b9-221d4065d1ad service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:25.153546 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:25.154246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:25.154583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:25.154806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:25.164277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:25.165630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:25.175840 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1206/2409] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:26.634329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210889-4a73-4726-a392-2b1f9f78ab83 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:26.634875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210889-4a73-4726-a392-2b1f9f78ab83 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:26.634875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03210889-4a73-4726-a392-2b1f9f78ab83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:26.654993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-03210889-4a73-4726-a392-2b1f9f78ab83 None None] MFA Rules not processed for user `b750b0c7df7f4cee94b5b4d578f601b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:26.656322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-03210889-4a73-4726-a392-2b1f9f78ab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:26.664651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1204/2410] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:26.916013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:26:26.918394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:26.931978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed4982-9017-4544-a5a7-832a0bbcd586 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:26:26.936106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8d0f5-c1e0-4b60-93a1-0055cbd62734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:26:26.936929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8d0f5-c1e0-4b60-93a1-0055cbd62734 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:26.937217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8d0f5-c1e0-4b60-93a1-0055cbd62734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:26.937478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8d0f5-c1e0-4b60-93a1-0055cbd62734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:26.939432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-07b8d0f5-c1e0-4b60-93a1-0055cbd62734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:26.949138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8d0f5-c1e0-4b60-93a1-0055cbd62734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:26:26.950596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8d0f5-c1e0-4b60-93a1-0055cbd62734 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:26:26.959259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1207/2411] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:26:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:26:36.648668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=39 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:36.955685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:26:42.586782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26b804-8934-47df-8c11-bfb0ec1b99c8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:26:42.587328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26b804-8934-47df-8c11-bfb0ec1b99c8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:26:42.587328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26b804-8934-47df-8c11-bfb0ec1b99c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:26:42.605860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-bb26b804-8934-47df-8c11-bfb0ec1b99c8 None None] MFA Rules not processed for user `b750b0c7df7f4cee94b5b4d578f601b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:26:42.607172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bb26b804-8934-47df-8c11-bfb0ec1b99c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:26:42.614758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1205/2412] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:26:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:26:52.600352 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:05.996961 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2260-d15d-4fc8-ad60-d4a330ae7339 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:05.998108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2260-d15d-4fc8-ad60-d4a330ae7339 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:05.998276 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b2260-d15d-4fc8-ad60-d4a330ae7339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:06.043065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-058b2260-d15d-4fc8-ad60-d4a330ae7339 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:06.044334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-058b2260-d15d-4fc8-ad60-d4a330ae7339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:06.159953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1208/2413] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:27:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:06.183889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26b804-8934-47df-8c11-bfb0ec1b99c8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:06.185612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-bb26b804-8934-47df-8c11-bfb0ec1b99c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:06.309127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26b804-8934-47df-8c11-bfb0ec1b99c8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:06.320219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81512c66-83e8-45d8-95a5-237a23a09c9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:06.320892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81512c66-83e8-45d8-95a5-237a23a09c9f service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:06.321160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81512c66-83e8-45d8-95a5-237a23a09c9f service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:06.321429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81512c66-83e8-45d8-95a5-237a23a09c9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:06.331332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81512c66-83e8-45d8-95a5-237a23a09c9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:06.335370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81512c66-83e8-45d8-95a5-237a23a09c9f service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:06.343956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1206/2414] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:27:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:27:09.346271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2260-d15d-4fc8-ad60-d4a330ae7339 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:09.383685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b2260-d15d-4fc8-ad60-d4a330ae7339 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:09.392139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:09.393576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:09.394127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:09.394681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:09.397135 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:09.455555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:09.457724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:09.491636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1209/2415] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:12.879997 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81512c66-83e8-45d8-95a5-237a23a09c9f service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:12.893626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81512c66-83e8-45d8-95a5-237a23a09c9f service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:12.899553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:12.900900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:12.901925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:12.902716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:12.909208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:12.932660 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:12.934442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:12.948141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1207/2416] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:13.598322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:13.609315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2e22b-68fe-4ad1-b1e4-c7bb873895e3 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:13.613937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:13.614937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:13.615485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:13.615976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:13.618087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:13.632773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:13.634649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:13.650596 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1210/2417] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:15.285745 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:15.287813 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:15.390775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:15.395370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:15.428602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef02c99-e52a-4f6f-b836-fd2b11488785 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:15.441753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:15.443742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:15.446650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:15.447454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] PATH_INFO: `/v3/users/a0210f69010b4529822bcdac1bdf8b8a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:15.451592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0210f69010b4529822bcdac1bdf8b8a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:15.454499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:15.488056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4fa587-b4df-49ee-a203-fe353f96dbfd service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:15.496283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:15.497023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:15.497303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:15.497624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:15.500909 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:15.521874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:15.525948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:15.543012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1208/2418] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:15.557891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0210f69010b4529822bcdac1bdf8b8a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:15.578329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0210f69010b4529822bcdac1bdf8b8a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:15.606932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:15.628822 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0210f69010b4529822bcdac1bdf8b8a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:15.642182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0210f69010b4529822bcdac1bdf8b8a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:15.643072 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aa825a9f-229d-4ea2-8887-5ed8ff729cd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0210f69010b4529822bcdac1bdf8b8a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:15.701722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1211/2419] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:15 2022] DELETE /identity/v3/users/a0210f69010b4529822bcdac1bdf8b8a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:16.293962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:16.352159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1d68a-8074-4880-aab9-00956519095c service placement] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:16.400975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:16.562263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:16.564514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:16.571694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:16.573684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] PATH_INFO: `/v3/projects/94c7507618b247c98e6bb541a48dbd31` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:16.577963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] RBAC: Authorizing `identity:delete_project(project_id=94c7507618b247c98e6bb541a48dbd31)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:16.582186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:16.718257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94c7507618b247c98e6bb541a48dbd31', 'request_id': 'req-33b8b7a6-94e3-4348-a1f3-bfd68939efda'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:16.728334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94c7507618b247c98e6bb541a48dbd31', 'request_id': 'req-33b8b7a6-94e3-4348-a1f3-bfd68939efda'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:16.751779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94c7507618b247c98e6bb541a48dbd31', 'request_id': 'req-33b8b7a6-94e3-4348-a1f3-bfd68939efda'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:16.791047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] The token cache is being invalidate because project 94c7507618b247c98e6bb541a48dbd31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:16.791523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:16.792458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-33b8b7a6-94e3-4348-a1f3-bfd68939efda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:16.799326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1209/2420] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:16 2022] DELETE /identity/v3/projects/94c7507618b247c98e6bb541a48dbd31 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:16.800806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1337-c436-419e-a06d-bf8c1806ae4e None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:16.801301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1337-c436-419e-a06d-bf8c1806ae4e None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:16.801384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb1337-c436-419e-a06d-bf8c1806ae4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:16.992589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3edb1337-c436-419e-a06d-bf8c1806ae4e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:16.994358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3edb1337-c436-419e-a06d-bf8c1806ae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:17.184936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1212/2421] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:17.193151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbd859-1bcc-4c62-925d-5dfbd7206ed4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:17.193443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbd859-1bcc-4c62-925d-5dfbd7206ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:17.193750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbd859-1bcc-4c62-925d-5dfbd7206ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:17.230739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-68bbd859-1bcc-4c62-925d-5dfbd7206ed4 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:17.234125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-68bbd859-1bcc-4c62-925d-5dfbd7206ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:17.255027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1210/2422] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:17.259435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb1337-c436-419e-a06d-bf8c1806ae4e None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:17.275000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb1337-c436-419e-a06d-bf8c1806ae4e None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:17.284663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:17.285383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:17.285682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:17.285960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:17.286713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:17.287031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:17.288660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:17.331684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1213/2423] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:27:17.336415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbd859-1bcc-4c62-925d-5dfbd7206ed4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:17.347639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbd859-1bcc-4c62-925d-5dfbd7206ed4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:17.365604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d5135-9ee6-476a-b5d7-6c55e9857e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:17.367971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d5135-9ee6-476a-b5d7-6c55e9857e5a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:17.367971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d5135-9ee6-476a-b5d7-6c55e9857e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:17.367971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d5135-9ee6-476a-b5d7-6c55e9857e5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:17.367971 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d5135-9ee6-476a-b5d7-6c55e9857e5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:17.371804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d5135-9ee6-476a-b5d7-6c55e9857e5a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:17.437332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1211/2424] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:17 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:17.441777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:17.456855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc048873-a610-493a-aa13-6d777d6611fe admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:17.606604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d1ef-3cf3-479d-8e41-811467afc357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:17.607525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d1ef-3cf3-479d-8e41-811467afc357 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:17.607667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d1ef-3cf3-479d-8e41-811467afc357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:17.607914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061d1ef-3cf3-479d-8e41-811467afc357 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:17.608943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061d1ef-3cf3-479d-8e41-811467afc357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:17.611364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061d1ef-3cf3-479d-8e41-811467afc357 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:17.659447 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1214/2425] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:17 2022] POST /identity/v3/users => generated 282 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:17.662113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d5135-9ee6-476a-b5d7-6c55e9857e5a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:17.687103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d5135-9ee6-476a-b5d7-6c55e9857e5a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:17.694372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f01ffc-106f-4e46-8ff2-b63e9bd4b373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:17.695285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f01ffc-106f-4e46-8ff2-b63e9bd4b373 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:17.695572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f01ffc-106f-4e46-8ff2-b63e9bd4b373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:17.695853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f01ffc-106f-4e46-8ff2-b63e9bd4b373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:17.696956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f01ffc-106f-4e46-8ff2-b63e9bd4b373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:17.698562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f01ffc-106f-4e46-8ff2-b63e9bd4b373 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:17.718058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1212/2426] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:17.719454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d1ef-3cf3-479d-8e41-811467afc357 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:17.734725 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061d1ef-3cf3-479d-8e41-811467afc357 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:17.746795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9d3c1-6e07-406a-9ec0-ad95c31e166c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:17.757449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d3c1-6e07-406a-9ec0-ad95c31e166c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:17.758030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d3c1-6e07-406a-9ec0-ad95c31e166c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:17.758302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9d3c1-6e07-406a-9ec0-ad95c31e166c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:17.759071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9d3c1-6e07-406a-9ec0-ad95c31e166c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:17.760743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9d3c1-6e07-406a-9ec0-ad95c31e166c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:17.781279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1215/2427] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:17.784177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f01ffc-106f-4e46-8ff2-b63e9bd4b373 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:17.797232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f01ffc-106f-4e46-8ff2-b63e9bd4b373 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:17.803039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70339ac-2192-43ef-b1b3-879097b09c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:17.804281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70339ac-2192-43ef-b1b3-879097b09c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:17.804668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70339ac-2192-43ef-b1b3-879097b09c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:17.805053 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70339ac-2192-43ef-b1b3-879097b09c9f admin admin] PATH_INFO: `/v3/projects/c9cdd7706ed243baa7033b27fbf046ea/users/88bf1f6867114bf585239802992e0e65/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:17.841699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70339ac-2192-43ef-b1b3-879097b09c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cdd7706ed243baa7033b27fbf046ea, user_id=88bf1f6867114bf585239802992e0e65, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:17.844564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70339ac-2192-43ef-b1b3-879097b09c9f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:17.868930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1213/2428] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:17 2022] PUT /identity/v3/projects/c9cdd7706ed243baa7033b27fbf046ea/users/88bf1f6867114bf585239802992e0e65/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:17.881005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4e6c5-1be3-4fc0-acbb-4bfb92407df5 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:17.883704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4e6c5-1be3-4fc0-acbb-4bfb92407df5 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:17.884538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4e6c5-1be3-4fc0-acbb-4bfb92407df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:17.997037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:18.004858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-43e4e6c5-1be3-4fc0-acbb-4bfb92407df5 None None] MFA Rules not processed for user `88bf1f6867114bf585239802992e0e65`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:18.009236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-43e4e6c5-1be3-4fc0-acbb-4bfb92407df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:18.142599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1216/2429] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:17 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:18.144877 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70339ac-2192-43ef-b1b3-879097b09c9f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:18.250418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:18.275892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70339ac-2192-43ef-b1b3-879097b09c9f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:18.283200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039c264-c28c-4ba8-aac5-d375513b5f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:18.284460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c264-c28c-4ba8-aac5-d375513b5f37 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:18.284907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c264-c28c-4ba8-aac5-d375513b5f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:18.286316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039c264-c28c-4ba8-aac5-d375513b5f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:18.287144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039c264-c28c-4ba8-aac5-d375513b5f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:18.291383 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039c264-c28c-4ba8-aac5-d375513b5f37 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:18.374959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1214/2430] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:18 2022] POST /identity/v3/projects => generated 363 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:18.376220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4e6c5-1be3-4fc0-acbb-4bfb92407df5 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:18.388618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4e6c5-1be3-4fc0-acbb-4bfb92407df5 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:18.536431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27a297-bc88-4a45-a9e6-3b4d0461c833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:18.537442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27a297-bc88-4a45-a9e6-3b4d0461c833 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:18.537871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27a297-bc88-4a45-a9e6-3b4d0461c833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:18.538274 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27a297-bc88-4a45-a9e6-3b4d0461c833 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:18.539083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27a297-bc88-4a45-a9e6-3b4d0461c833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:18.541921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27a297-bc88-4a45-a9e6-3b4d0461c833 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:18.606031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1217/2431] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:18 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:18.611981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039c264-c28c-4ba8-aac5-d375513b5f37 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:18.657507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039c264-c28c-4ba8-aac5-d375513b5f37 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:18.664961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32130c46-6c98-4f51-bc26-c543ba0b2beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:18.665915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130c46-6c98-4f51-bc26-c543ba0b2beb admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:18.666553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130c46-6c98-4f51-bc26-c543ba0b2beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:18.667125 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32130c46-6c98-4f51-bc26-c543ba0b2beb admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:18.668637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32130c46-6c98-4f51-bc26-c543ba0b2beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:18.709765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32130c46-6c98-4f51-bc26-c543ba0b2beb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:18.749832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1215/2432] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:18.753970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27a297-bc88-4a45-a9e6-3b4d0461c833 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:18.788585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27a297-bc88-4a45-a9e6-3b4d0461c833 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:18.810021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796be7-6b5f-49cb-9b19-66b4a53839f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:18.811622 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796be7-6b5f-49cb-9b19-66b4a53839f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:18.811905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796be7-6b5f-49cb-9b19-66b4a53839f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:18.812182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf796be7-6b5f-49cb-9b19-66b4a53839f5 admin admin] PATH_INFO: `/v3/projects/e3a128add7aa422385671d8b869044e5/users/af83fbc7049e4340b05e09eea14ba516/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:18.862192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf796be7-6b5f-49cb-9b19-66b4a53839f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a128add7aa422385671d8b869044e5, user_id=af83fbc7049e4340b05e09eea14ba516, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:18.865118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf796be7-6b5f-49cb-9b19-66b4a53839f5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:18.893343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1218/2433] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:18 2022] PUT /identity/v3/projects/e3a128add7aa422385671d8b869044e5/users/af83fbc7049e4340b05e09eea14ba516/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:18.897321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32130c46-6c98-4f51-bc26-c543ba0b2beb admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:18.941731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32130c46-6c98-4f51-bc26-c543ba0b2beb admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:19.121095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2102b997-21d8-4aa0-a0af-b6cdc0b3f219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:19.122194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b997-21d8-4aa0-a0af-b6cdc0b3f219 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:19.123361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b997-21d8-4aa0-a0af-b6cdc0b3f219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:19.123810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102b997-21d8-4aa0-a0af-b6cdc0b3f219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:19.124776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102b997-21d8-4aa0-a0af-b6cdc0b3f219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:19.130184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2102b997-21d8-4aa0-a0af-b6cdc0b3f219 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:19.173316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1216/2434] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:19.173552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796be7-6b5f-49cb-9b19-66b4a53839f5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:19.188120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf796be7-6b5f-49cb-9b19-66b4a53839f5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:19.202140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:19.203448 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:19.207175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:19.207613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] PATH_INFO: `/v3/projects/e3a128add7aa422385671d8b869044e5/users/af83fbc7049e4340b05e09eea14ba516/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:19.213839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a128add7aa422385671d8b869044e5, user_id=af83fbc7049e4340b05e09eea14ba516, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:19.220319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:19.242537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1219/2435] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:19 2022] PUT /identity/v3/projects/e3a128add7aa422385671d8b869044e5/users/af83fbc7049e4340b05e09eea14ba516/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:19.256234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91499c40-bd6c-4031-9553-e5a77e980dcb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:19.257517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91499c40-bd6c-4031-9553-e5a77e980dcb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:19.257888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91499c40-bd6c-4031-9553-e5a77e980dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:19.330288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-91499c40-bd6c-4031-9553-e5a77e980dcb None None] MFA Rules not processed for user `af83fbc7049e4340b05e09eea14ba516`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:19.333156 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-91499c40-bd6c-4031-9553-e5a77e980dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:19.350046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:19.354310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:19.533884 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:19.546893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ceff2-2b42-4ef4-a78c-17d4bd3ea217 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:19.551031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1217/2436] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:19 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:19.563656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:19.565307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91499c40-bd6c-4031-9553-e5a77e980dcb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:19.566268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:19.566590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:19.566870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] PATH_INFO: `/v3/users/5ca8be8093b547ee953ea646f90fdcae` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:19.568418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-91499c40-bd6c-4031-9553-e5a77e980dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:19.568734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca8be8093b547ee953ea646f90fdcae)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:19.573409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:19.626863 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:19.662979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91499c40-bd6c-4031-9553-e5a77e980dcb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:19.668109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82c204-f2bb-4807-a61f-f94e9f72fc8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:19.668963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82c204-f2bb-4807-a61f-f94e9f72fc8a service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:19.669570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82c204-f2bb-4807-a61f-f94e9f72fc8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:19.669952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82c204-f2bb-4807-a61f-f94e9f72fc8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:19.682021 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82c204-f2bb-4807-a61f-f94e9f72fc8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:19.685432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca8be8093b547ee953ea646f90fdcae'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:19.692670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e82c204-f2bb-4807-a61f-f94e9f72fc8a service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:19.695654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca8be8093b547ee953ea646f90fdcae'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:19.704702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1218/2437] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:27:19.713411 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca8be8093b547ee953ea646f90fdcae'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:19.714531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca8be8093b547ee953ea646f90fdcae'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:19.715398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca8be8093b547ee953ea646f90fdcae'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:19.738330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1220/2438] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:19 2022] DELETE /identity/v3/users/5ca8be8093b547ee953ea646f90fdcae => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:19.842161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82c204-f2bb-4807-a61f-f94e9f72fc8a service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:19.860617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82c204-f2bb-4807-a61f-f94e9f72fc8a service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:20.007027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:20.011946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:20.015438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:20.015876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:20.029240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:20.090645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:20.093457 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:20.191956 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:20.194632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:20.212263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1219/2439] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:19 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:20.256853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a2b6f-6f01-4cef-9955-3f2880307fdd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:20.264933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:20.266391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:20.268767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:20.269361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] PATH_INFO: `/v3/users/3aab24bab8d74b7aa5a20e7b2b89b5d7` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:20.270927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aab24bab8d74b7aa5a20e7b2b89b5d7)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:20.273044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:20.412167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:20.416048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:20.451208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aab24bab8d74b7aa5a20e7b2b89b5d7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.460872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976b157-9999-44a4-8fb5-f1540129c190 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:20.464734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aab24bab8d74b7aa5a20e7b2b89b5d7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.475502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:20.476532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:20.477041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:20.477469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] PATH_INFO: `/v3/projects/e1166ac77e8e403aa5f81e452bc039ac` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:20.479232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1166ac77e8e403aa5f81e452bc039ac)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:20.481200 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:20.522017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aab24bab8d74b7aa5a20e7b2b89b5d7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.522890 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aab24bab8d74b7aa5a20e7b2b89b5d7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.523797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9836653b-dc03-41cd-a704-da7af73125c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aab24bab8d74b7aa5a20e7b2b89b5d7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.538738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:20.560996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1221/2440] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:20 2022] DELETE /identity/v3/users/3aab24bab8d74b7aa5a20e7b2b89b5d7 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:20.568353 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1166ac77e8e403aa5f81e452bc039ac', 'request_id': 'req-8ff329af-523f-480b-a363-d897dd526945'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.577454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1166ac77e8e403aa5f81e452bc039ac', 'request_id': 'req-8ff329af-523f-480b-a363-d897dd526945'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.586473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1166ac77e8e403aa5f81e452bc039ac', 'request_id': 'req-8ff329af-523f-480b-a363-d897dd526945'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.603684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] The token cache is being invalidate because project e1166ac77e8e403aa5f81e452bc039ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:20.603978 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.604668 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8ff329af-523f-480b-a363-d897dd526945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.606328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1220/2441] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:20 2022] DELETE /identity/v3/projects/e1166ac77e8e403aa5f81e452bc039ac => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:20.612328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2c136-b206-42d1-a7d4-d6295f2d3643 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:20.612803 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2c136-b206-42d1-a7d4-d6295f2d3643 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:20.613294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2c136-b206-42d1-a7d4-d6295f2d3643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:20.654422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-07c2c136-b206-42d1-a7d4-d6295f2d3643 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:20.655723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-07c2c136-b206-42d1-a7d4-d6295f2d3643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:20.744800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1222/2442] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:20.755135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9486d1-08c7-4261-82ef-4bb21bf790a9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:20.755489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9486d1-08c7-4261-82ef-4bb21bf790a9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:20.755777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9486d1-08c7-4261-82ef-4bb21bf790a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:20.795871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6a9486d1-08c7-4261-82ef-4bb21bf790a9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:20.797764 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6a9486d1-08c7-4261-82ef-4bb21bf790a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:20.813213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2c136-b206-42d1-a7d4-d6295f2d3643 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:20.820087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1221/2443] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:20.821252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9486d1-08c7-4261-82ef-4bb21bf790a9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:20.824623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6a9486d1-08c7-4261-82ef-4bb21bf790a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:20.833339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2c136-b206-42d1-a7d4-d6295f2d3643 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:20.843809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:20.844508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:20.844790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:20.845064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:20.845845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:20.846339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:20.848722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:20.856665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9486d1-08c7-4261-82ef-4bb21bf790a9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:20.876183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1223/2444] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:27:20.877136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:20.881205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:20.882206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:20.882487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:20.882715 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] PATH_INFO: `/v3/projects/e6a24dfb85b04adfa565f06fde95d6b6` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:20.884998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a24dfb85b04adfa565f06fde95d6b6)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:20.889166 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:20.891933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fb623-aae2-4315-86dc-8b9d8ce0d106 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:20.892673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:20.899462 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b59113-2ca3-43c4-941e-1b17713f0b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:20.900312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b59113-2ca3-43c4-941e-1b17713f0b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:20.900626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b59113-2ca3-43c4-941e-1b17713f0b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:20.901189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b59113-2ca3-43c4-941e-1b17713f0b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:20.901983 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b59113-2ca3-43c4-941e-1b17713f0b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:20.904415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b59113-2ca3-43c4-941e-1b17713f0b9c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:20.937696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1224/2445] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:20 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:20.941806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b59113-2ca3-43c4-941e-1b17713f0b9c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:20.954868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a24dfb85b04adfa565f06fde95d6b6', 'request_id': 'req-1dc49e50-9020-40b3-999e-8cd659ff7461'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.964106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a24dfb85b04adfa565f06fde95d6b6', 'request_id': 'req-1dc49e50-9020-40b3-999e-8cd659ff7461'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:20.979921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a24dfb85b04adfa565f06fde95d6b6', 'request_id': 'req-1dc49e50-9020-40b3-999e-8cd659ff7461'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:21.005265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] The token cache is being invalidate because project e6a24dfb85b04adfa565f06fde95d6b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:21.005680 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:21.006426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1dc49e50-9020-40b3-999e-8cd659ff7461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:21.008436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1222/2446] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:20 2022] DELETE /identity/v3/projects/e6a24dfb85b04adfa565f06fde95d6b6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:21.014229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd1aa4-f3c2-4325-bd0b-da1368421e25 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.014490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd1aa4-f3c2-4325-bd0b-da1368421e25 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.014794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd1aa4-f3c2-4325-bd0b-da1368421e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.035786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a5cd1aa4-f3c2-4325-bd0b-da1368421e25 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:21.037159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a5cd1aa4-f3c2-4325-bd0b-da1368421e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:21.052891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b59113-2ca3-43c4-941e-1b17713f0b9c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.125198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1223/2447] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:21.126479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.127368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.127655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.127936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.128639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.130738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.139131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430deb64-8af0-4033-a527-6e693e05d37e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.144848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430deb64-8af0-4033-a527-6e693e05d37e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.144848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430deb64-8af0-4033-a527-6e693e05d37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.150711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:21.167689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1225/2448] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:20 2022] POST /identity/v3/users => generated 285 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:21.173067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.177184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:21.182540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-430deb64-8af0-4033-a527-6e693e05d37e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:21.183794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-430deb64-8af0-4033-a527-6e693e05d37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:21.195123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0801f2-5fee-400c-b099-0c2ab5b767ee admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.198550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1224/2449] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:21.201422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162c311-4fbb-450c-a374-74be3b992a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.202159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c311-4fbb-450c-a374-74be3b992a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.203466 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430deb64-8af0-4033-a527-6e693e05d37e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.205960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c311-4fbb-450c-a374-74be3b992a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.206755 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c311-4fbb-450c-a374-74be3b992a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.207789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162c311-4fbb-450c-a374-74be3b992a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.209648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162c311-4fbb-450c-a374-74be3b992a09 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.214619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430deb64-8af0-4033-a527-6e693e05d37e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.222475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.223184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.223612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.223896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.224663 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:21.225005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.226799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.230190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1226/2450] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:21.231976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162c311-4fbb-450c-a374-74be3b992a09 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.241462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1225/2451] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:27:21.251261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162c311-4fbb-450c-a374-74be3b992a09 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.252025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.257851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903a7f1-4e19-42be-bb63-6e47125372c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.258687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a7f1-4e19-42be-bb63-6e47125372c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.259143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a7f1-4e19-42be-bb63-6e47125372c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.259397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903a7f1-4e19-42be-bb63-6e47125372c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.260487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903a7f1-4e19-42be-bb63-6e47125372c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.262414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903a7f1-4e19-42be-bb63-6e47125372c2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.266181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437831d9-a31f-4897-a085-9d6881bb9822 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.271481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7b31c-7bf8-49a1-ba9c-ecad493fd43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.272298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7b31c-7bf8-49a1-ba9c-ecad493fd43c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.272687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7b31c-7bf8-49a1-ba9c-ecad493fd43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.273093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e7b31c-7bf8-49a1-ba9c-ecad493fd43c admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.273954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7b31c-7bf8-49a1-ba9c-ecad493fd43c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.276851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e7b31c-7bf8-49a1-ba9c-ecad493fd43c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.282300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1227/2452] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:21.283878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903a7f1-4e19-42be-bb63-6e47125372c2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.297761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903a7f1-4e19-42be-bb63-6e47125372c2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.302959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505e5dc-1030-4029-9232-8bf08754ea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.303872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e5dc-1030-4029-9232-8bf08754ea93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.304336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e5dc-1030-4029-9232-8bf08754ea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.304621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505e5dc-1030-4029-9232-8bf08754ea93 admin admin] PATH_INFO: `/v3/projects/33fbb6920f374f9a9381f06774d00b1a/users/ae349082e19443999ec07ede3d90b0a6/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.310232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1226/2453] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:21.314627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7b31c-7bf8-49a1-ba9c-ecad493fd43c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.325386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505e5dc-1030-4029-9232-8bf08754ea93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33fbb6920f374f9a9381f06774d00b1a, user_id=ae349082e19443999ec07ede3d90b0a6, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.328770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505e5dc-1030-4029-9232-8bf08754ea93 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.349361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1228/2454] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:21 2022] PUT /identity/v3/projects/33fbb6920f374f9a9381f06774d00b1a/users/ae349082e19443999ec07ede3d90b0a6/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:21.363044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c571-934e-44b7-87bc-16715f55bad9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.363515 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c571-934e-44b7-87bc-16715f55bad9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.363879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5c571-934e-44b7-87bc-16715f55bad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.381650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:21.396361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e7b31c-7bf8-49a1-ba9c-ecad493fd43c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.415029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e4e5c571-934e-44b7-87bc-16715f55bad9 None None] MFA Rules not processed for user `ae349082e19443999ec07ede3d90b0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:21.416349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e4e5c571-934e-44b7-87bc-16715f55bad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:21.465846 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1096e8-e0f6-4810-97c5-c71191d8250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.467164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1096e8-e0f6-4810-97c5-c71191d8250a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.467654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1096e8-e0f6-4810-97c5-c71191d8250a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.468169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1096e8-e0f6-4810-97c5-c71191d8250a admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.469173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1096e8-e0f6-4810-97c5-c71191d8250a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.471633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1096e8-e0f6-4810-97c5-c71191d8250a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.491849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1229/2455] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:21.496418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5c571-934e-44b7-87bc-16715f55bad9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.497706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1227/2456] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:21.501760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1096e8-e0f6-4810-97c5-c71191d8250a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.513050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5c571-934e-44b7-87bc-16715f55bad9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.514121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1096e8-e0f6-4810-97c5-c71191d8250a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.517781 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95b958-1bc6-4b6e-a3c2-a979b3cc3cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.518668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b958-1bc6-4b6e-a3c2-a979b3cc3cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.519028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b958-1bc6-4b6e-a3c2-a979b3cc3cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.519380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b958-1bc6-4b6e-a3c2-a979b3cc3cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.520154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95b958-1bc6-4b6e-a3c2-a979b3cc3cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.520912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2628d-ddac-48fd-a53e-017cdafd4c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.522342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95b958-1bc6-4b6e-a3c2-a979b3cc3cf9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.536107 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2628d-ddac-48fd-a53e-017cdafd4c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.540249 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2628d-ddac-48fd-a53e-017cdafd4c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.540649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2628d-ddac-48fd-a53e-017cdafd4c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.541774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2628d-ddac-48fd-a53e-017cdafd4c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.545702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2628d-ddac-48fd-a53e-017cdafd4c7a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.568262 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1230/2457] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:21.569128 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95b958-1bc6-4b6e-a3c2-a979b3cc3cf9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.585590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1228/2458] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:21.586601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2628d-ddac-48fd-a53e-017cdafd4c7a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.610426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2628d-ddac-48fd-a53e-017cdafd4c7a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.653782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95b958-1bc6-4b6e-a3c2-a979b3cc3cf9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.659888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649467-606c-4403-8a12-c99bc36b5740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.660860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649467-606c-4403-8a12-c99bc36b5740 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.661159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649467-606c-4403-8a12-c99bc36b5740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.661488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0649467-606c-4403-8a12-c99bc36b5740 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.662224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649467-606c-4403-8a12-c99bc36b5740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.664495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0649467-606c-4403-8a12-c99bc36b5740 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.697118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1231/2459] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/users => generated 290 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:21.698232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649467-606c-4403-8a12-c99bc36b5740 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.714042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0649467-606c-4403-8a12-c99bc36b5740 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.720721 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ebccd-dffb-414c-8083-a8b564a33fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.721573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ebccd-dffb-414c-8083-a8b564a33fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.722228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ebccd-dffb-414c-8083-a8b564a33fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.723000 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ebccd-dffb-414c-8083-a8b564a33fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.723364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ebccd-dffb-414c-8083-a8b564a33fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.725008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ebccd-dffb-414c-8083-a8b564a33fc7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.726697 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:21.742364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1232/2460] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:21.748499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ebccd-dffb-414c-8083-a8b564a33fc7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.751690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857f99-5fa1-4910-a1b6-1bd0d666b9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.752398 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857f99-5fa1-4910-a1b6-1bd0d666b9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.752678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857f99-5fa1-4910-a1b6-1bd0d666b9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.752976 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c857f99-5fa1-4910-a1b6-1bd0d666b9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.753850 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c857f99-5fa1-4910-a1b6-1bd0d666b9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.755495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c857f99-5fa1-4910-a1b6-1bd0d666b9aa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.766381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ebccd-dffb-414c-8083-a8b564a33fc7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.766635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1229/2461] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:21.771050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857f99-5fa1-4910-a1b6-1bd0d666b9aa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.776171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998de79-0de2-467f-9f1e-9e5f8b30ad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.777267 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998de79-0de2-467f-9f1e-9e5f8b30ad6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.777627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998de79-0de2-467f-9f1e-9e5f8b30ad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.778001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998de79-0de2-467f-9f1e-9e5f8b30ad6f admin admin] PATH_INFO: `/v3/projects/ef72a4cdc79e42f5b89bd2af9e08e685/users/8f3d9b87c3bb40a788516bf41abf32fb/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.786083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c857f99-5fa1-4910-a1b6-1bd0d666b9aa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.793582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fde2c-e093-49bb-9f99-6faaa86d83ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.795321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fde2c-e093-49bb-9f99-6faaa86d83ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.795881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fde2c-e093-49bb-9f99-6faaa86d83ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.796270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fde2c-e093-49bb-9f99-6faaa86d83ff admin admin] PATH_INFO: `/v3/projects/136509c568a64771b7617842464aab9f/users/e6d2c4a6ec604c2c88ec349a3802e7a0/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.802392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998de79-0de2-467f-9f1e-9e5f8b30ad6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef72a4cdc79e42f5b89bd2af9e08e685, user_id=8f3d9b87c3bb40a788516bf41abf32fb, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.805126 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998de79-0de2-467f-9f1e-9e5f8b30ad6f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.815639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fde2c-e093-49bb-9f99-6faaa86d83ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=136509c568a64771b7617842464aab9f, user_id=e6d2c4a6ec604c2c88ec349a3802e7a0, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.819140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75fde2c-e093-49bb-9f99-6faaa86d83ff admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.838847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1230/2462] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:21 2022] PUT /identity/v3/projects/136509c568a64771b7617842464aab9f/users/e6d2c4a6ec604c2c88ec349a3802e7a0/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:21.843496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1233/2463] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:21 2022] PUT /identity/v3/projects/ef72a4cdc79e42f5b89bd2af9e08e685/users/8f3d9b87c3bb40a788516bf41abf32fb/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:21.848188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fde2c-e093-49bb-9f99-6faaa86d83ff admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.849294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d505c5-2e90-4cc2-8a98-d77650c5e08a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.849738 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d505c5-2e90-4cc2-8a98-d77650c5e08a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.850266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d505c5-2e90-4cc2-8a98-d77650c5e08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.888183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-e4d505c5-2e90-4cc2-8a98-d77650c5e08a None None] MFA Rules not processed for user `e6d2c4a6ec604c2c88ec349a3802e7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:21.889386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e4d505c5-2e90-4cc2-8a98-d77650c5e08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:21.946054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:21.969093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fde2c-e093-49bb-9f99-6faaa86d83ff admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.975019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1234/2464] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:21.976100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c82e2-ef61-4a8a-ae51-7f075e9b0548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.977005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c82e2-ef61-4a8a-ae51-7f075e9b0548 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:21.977388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c82e2-ef61-4a8a-ae51-7f075e9b0548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:21.978210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c82e2-ef61-4a8a-ae51-7f075e9b0548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:21.979184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c82e2-ef61-4a8a-ae51-7f075e9b0548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:21.981160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d505c5-2e90-4cc2-8a98-d77650c5e08a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:21.982783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c82e2-ef61-4a8a-ae51-7f075e9b0548 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:21.992563 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d505c5-2e90-4cc2-8a98-d77650c5e08a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:21.999076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857632-42e5-468f-8afe-6957dabebb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:21.999816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857632-42e5-468f-8afe-6957dabebb80 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.000102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857632-42e5-468f-8afe-6957dabebb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.000383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5857632-42e5-468f-8afe-6957dabebb80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.001092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5857632-42e5-468f-8afe-6957dabebb80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:22.003120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5857632-42e5-468f-8afe-6957dabebb80 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:22.016181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1231/2465] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:22.017539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c82e2-ef61-4a8a-ae51-7f075e9b0548 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:22.036098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c82e2-ef61-4a8a-ae51-7f075e9b0548 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:22.042494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1235/2466] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:21 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:22.043163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857632-42e5-468f-8afe-6957dabebb80 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:22.131549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0760f0e-a6fb-48ce-8356-23d38cc4e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:22.134425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760f0e-a6fb-48ce-8356-23d38cc4e8fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.135013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760f0e-a6fb-48ce-8356-23d38cc4e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.135743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0760f0e-a6fb-48ce-8356-23d38cc4e8fd admin admin] PATH_INFO: `/v3/projects/ef72a4cdc79e42f5b89bd2af9e08e685/users/8f3d9b87c3bb40a788516bf41abf32fb/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.140046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0760f0e-a6fb-48ce-8356-23d38cc4e8fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef72a4cdc79e42f5b89bd2af9e08e685, user_id=8f3d9b87c3bb40a788516bf41abf32fb, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:22.144047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0760f0e-a6fb-48ce-8356-23d38cc4e8fd admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:22.161339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1232/2467] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:22 2022] PUT /identity/v3/projects/ef72a4cdc79e42f5b89bd2af9e08e685/users/8f3d9b87c3bb40a788516bf41abf32fb/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:22.174442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a29159-dfa4-4029-b2a7-9771d7fbe380 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.174908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a29159-dfa4-4029-b2a7-9771d7fbe380 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.175348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a29159-dfa4-4029-b2a7-9771d7fbe380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.208754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5857632-42e5-468f-8afe-6957dabebb80 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:22.224479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f7a29159-dfa4-4029-b2a7-9771d7fbe380 None None] MFA Rules not processed for user `8f3d9b87c3bb40a788516bf41abf32fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:22.225761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f7a29159-dfa4-4029-b2a7-9771d7fbe380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:22.286091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:22.317531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98196940-fd70-4374-a876-44fdc58ba339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:22.317687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98196940-fd70-4374-a876-44fdc58ba339 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.317976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98196940-fd70-4374-a876-44fdc58ba339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.318198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98196940-fd70-4374-a876-44fdc58ba339 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.318898 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98196940-fd70-4374-a876-44fdc58ba339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:22.320844 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98196940-fd70-4374-a876-44fdc58ba339 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:22.333597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1233/2468] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:22.346882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a29159-dfa4-4029-b2a7-9771d7fbe380 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:22.352253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f7a29159-dfa4-4029-b2a7-9771d7fbe380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:22.357403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1236/2469] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:22 2022] POST /identity/v3/users => generated 295 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:22.358535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98196940-fd70-4374-a876-44fdc58ba339 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:22.370469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98196940-fd70-4374-a876-44fdc58ba339 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:22.376951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f544503-f3f5-4a5b-af00-83122696ddc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:22.377648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544503-f3f5-4a5b-af00-83122696ddc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.377990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544503-f3f5-4a5b-af00-83122696ddc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.378271 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f544503-f3f5-4a5b-af00-83122696ddc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.379022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f544503-f3f5-4a5b-af00-83122696ddc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:22.380652 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f544503-f3f5-4a5b-af00-83122696ddc1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:22.398200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1237/2470] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:22.399165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f544503-f3f5-4a5b-af00-83122696ddc1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:22.410181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f544503-f3f5-4a5b-af00-83122696ddc1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:22.416730 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eec98a-f580-4e10-85f8-aad1c5c01e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:22.417636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec98a-f580-4e10-85f8-aad1c5c01e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.417979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec98a-f580-4e10-85f8-aad1c5c01e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.418510 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eec98a-f580-4e10-85f8-aad1c5c01e2e admin admin] PATH_INFO: `/v3/projects/49f022fa9fd4474280ddde89aff11c22/users/ed7589d70ed64394b2e5ab12783feca8/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.438047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eec98a-f580-4e10-85f8-aad1c5c01e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f022fa9fd4474280ddde89aff11c22, user_id=ed7589d70ed64394b2e5ab12783feca8, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:22.441197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eec98a-f580-4e10-85f8-aad1c5c01e2e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:22.456736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a29159-dfa4-4029-b2a7-9771d7fbe380 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:22.472314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1238/2471] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:22 2022] PUT /identity/v3/projects/49f022fa9fd4474280ddde89aff11c22/users/ed7589d70ed64394b2e5ab12783feca8/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:22.477476 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eec98a-f580-4e10-85f8-aad1c5c01e2e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:22.488722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:22.541066 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c093d4-4048-4db8-9124-9eab53210aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:22.542035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c093d4-4048-4db8-9124-9eab53210aed service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.542883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c093d4-4048-4db8-9124-9eab53210aed service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.542883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c093d4-4048-4db8-9124-9eab53210aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.617282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eec98a-f580-4e10-85f8-aad1c5c01e2e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:22.628776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf236-e5dc-4c44-8ed1-f55b5831841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:22.629648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf236-e5dc-4c44-8ed1-f55b5831841c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.629812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf236-e5dc-4c44-8ed1-f55b5831841c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.630110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccf236-e5dc-4c44-8ed1-f55b5831841c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.635068 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccf236-e5dc-4c44-8ed1-f55b5831841c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:22.640548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccf236-e5dc-4c44-8ed1-f55b5831841c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:22.656586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1239/2472] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:22.661053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf236-e5dc-4c44-8ed1-f55b5831841c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:22.674107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccf236-e5dc-4c44-8ed1-f55b5831841c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:22.675367 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c093d4-4048-4db8-9124-9eab53210aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:22.678609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c093d4-4048-4db8-9124-9eab53210aed service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:22.680461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66488cc-3cf2-4ffe-b3f3-492ff2d250b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:22.681383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66488cc-3cf2-4ffe-b3f3-492ff2d250b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.681874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66488cc-3cf2-4ffe-b3f3-492ff2d250b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.682156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66488cc-3cf2-4ffe-b3f3-492ff2d250b3 admin admin] PATH_INFO: `/v3/projects/49f022fa9fd4474280ddde89aff11c22/users/ed7589d70ed64394b2e5ab12783feca8/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.684248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66488cc-3cf2-4ffe-b3f3-492ff2d250b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f022fa9fd4474280ddde89aff11c22, user_id=ed7589d70ed64394b2e5ab12783feca8, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:22.687522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66488cc-3cf2-4ffe-b3f3-492ff2d250b3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:22.696251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1234/2473] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:22 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:22.704484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1240/2474] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:22 2022] PUT /identity/v3/projects/49f022fa9fd4474280ddde89aff11c22/users/ed7589d70ed64394b2e5ab12783feca8/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:22.715481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10327c-0354-4524-a4ff-b2c353dc3a5a None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:22.715990 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10327c-0354-4524-a4ff-b2c353dc3a5a None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:22.716712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10327c-0354-4524-a4ff-b2c353dc3a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:22.754448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1a10327c-0354-4524-a4ff-b2c353dc3a5a None None] MFA Rules not processed for user `ed7589d70ed64394b2e5ab12783feca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:22.755970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1a10327c-0354-4524-a4ff-b2c353dc3a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:22.886879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1235/2475] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:22.899189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66488cc-3cf2-4ffe-b3f3-492ff2d250b3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:23.005437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66488cc-3cf2-4ffe-b3f3-492ff2d250b3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:23.023195 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcd81b-c66a-49ac-a240-7d73a9855bb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:23.023959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd81b-c66a-49ac-a240-7d73a9855bb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:23.024317 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd81b-c66a-49ac-a240-7d73a9855bb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:23.024690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcd81b-c66a-49ac-a240-7d73a9855bb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:23.099644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:23.122893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcd81b-c66a-49ac-a240-7d73a9855bb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:23.124471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcd81b-c66a-49ac-a240-7d73a9855bb2 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:23.135330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1241/2476] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:24.956967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10327c-0354-4524-a4ff-b2c353dc3a5a None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:25.070936 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:25.080046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10327c-0354-4524-a4ff-b2c353dc3a5a None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:25.085925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:25.086747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:25.087033 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:25.087345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] PATH_INFO: `/v3/users/ae349082e19443999ec07ede3d90b0a6` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:25.088542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae349082e19443999ec07ede3d90b0a6)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:25.090702 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:25.159284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae349082e19443999ec07ede3d90b0a6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.169040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae349082e19443999ec07ede3d90b0a6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.183439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae349082e19443999ec07ede3d90b0a6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.184767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae349082e19443999ec07ede3d90b0a6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.185649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae349082e19443999ec07ede3d90b0a6'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.204140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1236/2477] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:24 2022] DELETE /identity/v3/users/ae349082e19443999ec07ede3d90b0a6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:25.210685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66869b-c7af-44cd-9901-adb53ce46484 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:25.213066 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66869b-c7af-44cd-9901-adb53ce46484 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:25.213921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66869b-c7af-44cd-9901-adb53ce46484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:25.250508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8b66869b-c7af-44cd-9901-adb53ce46484 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:25.252099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8b66869b-c7af-44cd-9901-adb53ce46484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:25.337939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1242/2478] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:25.354124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:25.355600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:25.454189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676971a-4811-4fd0-874f-20d6685b30d9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:25.458477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b1ed-2d1f-4cef-a491-c9153c581a1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:25.459231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b1ed-2d1f-4cef-a491-c9153c581a1c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:25.459734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b1ed-2d1f-4cef-a491-c9153c581a1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:25.460088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350b1ed-2d1f-4cef-a491-c9153c581a1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:25.472376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350b1ed-2d1f-4cef-a491-c9153c581a1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:25.473899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350b1ed-2d1f-4cef-a491-c9153c581a1c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:25.485312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1237/2479] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:25.841848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66869b-c7af-44cd-9901-adb53ce46484 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:25.842821 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8b66869b-c7af-44cd-9901-adb53ce46484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:25.855663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66869b-c7af-44cd-9901-adb53ce46484 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:25.861812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:25.862720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:25.863002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:25.863282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] PATH_INFO: `/v3/projects/33fbb6920f374f9a9381f06774d00b1a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:25.864527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=33fbb6920f374f9a9381f06774d00b1a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:25.866594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:25.922602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33fbb6920f374f9a9381f06774d00b1a', 'request_id': 'req-2ad469fd-9bdd-4025-8bad-4abf160bde7f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.932955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33fbb6920f374f9a9381f06774d00b1a', 'request_id': 'req-2ad469fd-9bdd-4025-8bad-4abf160bde7f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.939959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33fbb6920f374f9a9381f06774d00b1a', 'request_id': 'req-2ad469fd-9bdd-4025-8bad-4abf160bde7f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.951597 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] The token cache is being invalidate because project 33fbb6920f374f9a9381f06774d00b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:25.951994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.952847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:25.954694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1243/2480] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:25 2022] DELETE /identity/v3/projects/33fbb6920f374f9a9381f06774d00b1a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:25.958662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b1ed-2d1f-4cef-a491-c9153c581a1c service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:25.970568 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350b1ed-2d1f-4cef-a491-c9153c581a1c service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:26.033690 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:26.041214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:26.042083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:26.042740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:26.043528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] PATH_INFO: `/v3/users/8f3d9b87c3bb40a788516bf41abf32fb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:26.045812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3d9b87c3bb40a788516bf41abf32fb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:26.048219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:26.105704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3d9b87c3bb40a788516bf41abf32fb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.115074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3d9b87c3bb40a788516bf41abf32fb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.128926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3d9b87c3bb40a788516bf41abf32fb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.131121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3d9b87c3bb40a788516bf41abf32fb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.131924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b5696fb9-f620-4e0b-b451-cf2837fdafda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3d9b87c3bb40a788516bf41abf32fb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.149751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1238/2481] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:25 2022] DELETE /identity/v3/users/8f3d9b87c3bb40a788516bf41abf32fb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:26.491802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:26.493281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:26.508519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad469fd-9bdd-4025-8bad-4abf160bde7f admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:26.586511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:26.587400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:26.587684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:26.587960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] PATH_INFO: `/v3/projects/ef72a4cdc79e42f5b89bd2af9e08e685` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:26.589154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef72a4cdc79e42f5b89bd2af9e08e685)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:26.592338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:26.595141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:26.670720 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef72a4cdc79e42f5b89bd2af9e08e685', 'request_id': 'req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.682967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef72a4cdc79e42f5b89bd2af9e08e685', 'request_id': 'req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.691048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef72a4cdc79e42f5b89bd2af9e08e685', 'request_id': 'req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.707618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] The token cache is being invalidate because project ef72a4cdc79e42f5b89bd2af9e08e685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:26.708325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.709143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-eab2af8f-5f4b-4bd7-8b18-045c3fe740b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:26.711308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1244/2482] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:26 2022] DELETE /identity/v3/projects/ef72a4cdc79e42f5b89bd2af9e08e685 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:26.717697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1456acf-0b03-4d48-881e-4ab33f96f2cd None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:26.718077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1456acf-0b03-4d48-881e-4ab33f96f2cd None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:26.718306 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1456acf-0b03-4d48-881e-4ab33f96f2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:26.755499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a1456acf-0b03-4d48-881e-4ab33f96f2cd None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:26.756710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a1456acf-0b03-4d48-881e-4ab33f96f2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:26.833051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1239/2483] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:26.841437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2732d-0d07-4535-8df1-3fe35b8535b0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:26.841868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2732d-0d07-4535-8df1-3fe35b8535b0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:26.842038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c2732d-0d07-4535-8df1-3fe35b8535b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:26.866337 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-94c2732d-0d07-4535-8df1-3fe35b8535b0 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:26.867828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-94c2732d-0d07-4535-8df1-3fe35b8535b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:26.883412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1245/2484] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:26.884845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1456acf-0b03-4d48-881e-4ab33f96f2cd None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:26.895828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1456acf-0b03-4d48-881e-4ab33f96f2cd None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:26.900963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:26.901822 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:26.902162 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:26.902500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:26.903375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:26.903952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:26.905854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:26.923078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1240/2485] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:27:26.924042 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2732d-0d07-4535-8df1-3fe35b8535b0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:26.934306 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c2732d-0d07-4535-8df1-3fe35b8535b0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:26.939648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21170698-fe06-4424-a115-82191ee867a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:26.940343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21170698-fe06-4424-a115-82191ee867a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:26.940632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21170698-fe06-4424-a115-82191ee867a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:26.940913 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21170698-fe06-4424-a115-82191ee867a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:26.941624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21170698-fe06-4424-a115-82191ee867a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:26.943642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21170698-fe06-4424-a115-82191ee867a3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:26.975806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1246/2486] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:26 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:26.980560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:26.995009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4998a-7b8f-4747-86b2-e4018f0be698 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.079479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291493d-84b2-4aa7-b5e1-06758969136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.080185 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291493d-84b2-4aa7-b5e1-06758969136b admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.080476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291493d-84b2-4aa7-b5e1-06758969136b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.084807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b291493d-84b2-4aa7-b5e1-06758969136b admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.085570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b291493d-84b2-4aa7-b5e1-06758969136b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.087649 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b291493d-84b2-4aa7-b5e1-06758969136b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.097869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:27.114720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1241/2487] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:26 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:27.118614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21170698-fe06-4424-a115-82191ee867a3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.131728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21170698-fe06-4424-a115-82191ee867a3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.156785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3a601-7c13-4240-81e3-95874e951536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.157608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3a601-7c13-4240-81e3-95874e951536 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.158043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3a601-7c13-4240-81e3-95874e951536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.158389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3a601-7c13-4240-81e3-95874e951536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.159640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3a601-7c13-4240-81e3-95874e951536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.161028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3a601-7c13-4240-81e3-95874e951536 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.186882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1247/2488] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:27.191252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291493d-84b2-4aa7-b5e1-06758969136b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.202115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b291493d-84b2-4aa7-b5e1-06758969136b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.207539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada7f30-80a7-4ede-b178-b2baa8abebfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.208358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada7f30-80a7-4ede-b178-b2baa8abebfb admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.208551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada7f30-80a7-4ede-b178-b2baa8abebfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.208831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada7f30-80a7-4ede-b178-b2baa8abebfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.209584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada7f30-80a7-4ede-b178-b2baa8abebfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.211257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada7f30-80a7-4ede-b178-b2baa8abebfb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.222567 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1242/2489] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:27.226714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3a601-7c13-4240-81e3-95874e951536 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.238620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3a601-7c13-4240-81e3-95874e951536 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.244010 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396ba1-8e2c-4423-b527-8818636a8fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.244927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396ba1-8e2c-4423-b527-8818636a8fde admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.245209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396ba1-8e2c-4423-b527-8818636a8fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.245493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396ba1-8e2c-4423-b527-8818636a8fde admin admin] PATH_INFO: `/v3/projects/de1dc77abef544e0bf24a788fb899424/users/b5f2507bcde54d16826249975646b192/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.268374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396ba1-8e2c-4423-b527-8818636a8fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1dc77abef544e0bf24a788fb899424, user_id=b5f2507bcde54d16826249975646b192, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.271962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396ba1-8e2c-4423-b527-8818636a8fde admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.297493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1248/2490] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:27 2022] PUT /identity/v3/projects/de1dc77abef544e0bf24a788fb899424/users/b5f2507bcde54d16826249975646b192/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:27.309809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bda11-510a-47bc-be3c-304e3d1fbf0b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.310356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bda11-510a-47bc-be3c-304e3d1fbf0b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.310660 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59bda11-510a-47bc-be3c-304e3d1fbf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.345878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a59bda11-510a-47bc-be3c-304e3d1fbf0b None None] MFA Rules not processed for user `b5f2507bcde54d16826249975646b192`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:27.347307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a59bda11-510a-47bc-be3c-304e3d1fbf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:27.441308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1243/2491] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:27.445950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396ba1-8e2c-4423-b527-8818636a8fde admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.545181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396ba1-8e2c-4423-b527-8818636a8fde admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.554340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e71cc5-702f-4aee-80fe-279831e787c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.555657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e71cc5-702f-4aee-80fe-279831e787c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.556117 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e71cc5-702f-4aee-80fe-279831e787c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.556767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e71cc5-702f-4aee-80fe-279831e787c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.558332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e71cc5-702f-4aee-80fe-279831e787c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.562757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e71cc5-702f-4aee-80fe-279831e787c5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.604182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bda11-510a-47bc-be3c-304e3d1fbf0b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.604710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1249/2492] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:27 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:27.616762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59bda11-510a-47bc-be3c-304e3d1fbf0b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.689751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b03846-afa6-4dd3-aaa5-be36e045aec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.690527 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b03846-afa6-4dd3-aaa5-be36e045aec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.690860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b03846-afa6-4dd3-aaa5-be36e045aec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.691210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b03846-afa6-4dd3-aaa5-be36e045aec3 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.691972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b03846-afa6-4dd3-aaa5-be36e045aec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.694043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b03846-afa6-4dd3-aaa5-be36e045aec3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.728137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1244/2493] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:27 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:27.729601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e71cc5-702f-4aee-80fe-279831e787c5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.743373 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e71cc5-702f-4aee-80fe-279831e787c5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.749395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99482e-a267-4575-9f54-1bfe2cd0b2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.750302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99482e-a267-4575-9f54-1bfe2cd0b2af admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.750595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99482e-a267-4575-9f54-1bfe2cd0b2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.750874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99482e-a267-4575-9f54-1bfe2cd0b2af admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.751693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99482e-a267-4575-9f54-1bfe2cd0b2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.753379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99482e-a267-4575-9f54-1bfe2cd0b2af admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.765513 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1250/2494] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:27.769413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b03846-afa6-4dd3-aaa5-be36e045aec3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.781103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b03846-afa6-4dd3-aaa5-be36e045aec3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.787693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d9b8e-2bf4-4e3a-ada9-51c5634fe1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.788615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d9b8e-2bf4-4e3a-ada9-51c5634fe1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.788900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d9b8e-2bf4-4e3a-ada9-51c5634fe1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.789219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d9b8e-2bf4-4e3a-ada9-51c5634fe1c3 admin admin] PATH_INFO: `/v3/projects/72b0b142a5b14c0387aeec2d539be5d8/users/b8b4f733aed944c7ad37cf4b4655d3e4/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.806128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d9b8e-2bf4-4e3a-ada9-51c5634fe1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b0b142a5b14c0387aeec2d539be5d8, user_id=b8b4f733aed944c7ad37cf4b4655d3e4, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.808872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d9b8e-2bf4-4e3a-ada9-51c5634fe1c3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.826480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1245/2495] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:27 2022] PUT /identity/v3/projects/72b0b142a5b14c0387aeec2d539be5d8/users/b8b4f733aed944c7ad37cf4b4655d3e4/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:27.830347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99482e-a267-4575-9f54-1bfe2cd0b2af admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.836752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:27.845794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99482e-a267-4575-9f54-1bfe2cd0b2af admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.918925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcbb95-49e7-4381-b761-591773fb0cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.919664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcbb95-49e7-4381-b761-591773fb0cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.919951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcbb95-49e7-4381-b761-591773fb0cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.920327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcbb95-49e7-4381-b761-591773fb0cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.920955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdcbb95-49e7-4381-b761-591773fb0cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.922688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdcbb95-49e7-4381-b761-591773fb0cb7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.933940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1251/2496] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:27.938250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d9b8e-2bf4-4e3a-ada9-51c5634fe1c3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:27.949958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d9b8e-2bf4-4e3a-ada9-51c5634fe1c3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:27.955569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:27.956487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.956766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.957043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] PATH_INFO: `/v3/projects/72b0b142a5b14c0387aeec2d539be5d8/users/b8b4f733aed944c7ad37cf4b4655d3e4/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:27.959118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b0b142a5b14c0387aeec2d539be5d8, user_id=b8b4f733aed944c7ad37cf4b4655d3e4, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:27.961805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:27.974625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1246/2497] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:27 2022] PUT /identity/v3/projects/72b0b142a5b14c0387aeec2d539be5d8/users/b8b4f733aed944c7ad37cf4b4655d3e4/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:27.987762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0604-da10-4099-be06-971e28b08780 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:27.988165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0604-da10-4099-be06-971e28b08780 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:27.988344 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca0604-da10-4099-be06-971e28b08780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:28.017672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-faca0604-da10-4099-be06-971e28b08780 None None] MFA Rules not processed for user `b8b4f733aed944c7ad37cf4b4655d3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:28.018918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-faca0604-da10-4099-be06-971e28b08780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:28.097358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1252/2498] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:28.110256 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:28.112394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:28.144881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:28.206123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393b30-17db-4778-9610-b02aabde39f1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:28.210179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:28.210832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:28.211112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:28.211384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:28.274827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:28.276310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:28.286910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1247/2499] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:31.019074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e661b47-91e8-4136-99d9-0ac73c1ca1d4 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:31.019589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e661b47-91e8-4136-99d9-0ac73c1ca1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:31.019684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e661b47-91e8-4136-99d9-0ac73c1ca1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:31.039825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6e661b47-91e8-4136-99d9-0ac73c1ca1d4 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:31.042178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6e661b47-91e8-4136-99d9-0ac73c1ca1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:31.102987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1253/2500] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 18 14:27:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:31.116334 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:31.117854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:31.203786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c361381-b30e-422c-b059-3010e33b7dec service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:31.209392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:31.210081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:31.210364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:31.210884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:31.219472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e661b47-91e8-4136-99d9-0ac73c1ca1d4 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:31.220692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:31.222386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:31.228614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e661b47-91e8-4136-99d9-0ac73c1ca1d4 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:31.232080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1248/2501] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:27:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:27:31.232466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:31.233149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:31.233789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:31.234293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:31.236088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:31.266329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:31.306799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:31.308259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:31.320283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1254/2502] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:27:33.209199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:33.210643 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:33.265326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:33.282520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e6db5-1a62-477c-848e-3e13559d2452 service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:33.287143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:33.288023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:33.288308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:33.288585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] PATH_INFO: `/v3/users/b750b0c7df7f4cee94b5b4d578f601b0` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:33.289811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b750b0c7df7f4cee94b5b4d578f601b0)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:33.291791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:33.350244 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b750b0c7df7f4cee94b5b4d578f601b0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.361501 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b750b0c7df7f4cee94b5b4d578f601b0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.375030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b750b0c7df7f4cee94b5b4d578f601b0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.375780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b750b0c7df7f4cee94b5b4d578f601b0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.376438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5e76c064-40b6-4834-8ca3-8fbf261817c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b750b0c7df7f4cee94b5b4d578f601b0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.389516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1249/2503] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:33 2022] DELETE /identity/v3/users/b750b0c7df7f4cee94b5b4d578f601b0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:33.583087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:33.596976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8c6f3-7e3d-44d2-bc0f-79f5ad2d1c8e service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:33.660744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:33.661626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:33.661991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:33.662209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] PATH_INFO: `/v3/projects/3e1451a6c59a4ef0b0cfe3cc6b98c605` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:33.663397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e1451a6c59a4ef0b0cfe3cc6b98c605)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:33.665375 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:33.718805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e1451a6c59a4ef0b0cfe3cc6b98c605', 'request_id': 'req-b7e67988-67d7-44d7-807a-3bc8c72fa333'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.728793 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e1451a6c59a4ef0b0cfe3cc6b98c605', 'request_id': 'req-b7e67988-67d7-44d7-807a-3bc8c72fa333'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.736683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e1451a6c59a4ef0b0cfe3cc6b98c605', 'request_id': 'req-b7e67988-67d7-44d7-807a-3bc8c72fa333'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.755633 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] The token cache is being invalidate because project 3e1451a6c59a4ef0b0cfe3cc6b98c605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:33.755921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.756628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-b7e67988-67d7-44d7-807a-3bc8c72fa333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:33.758290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1255/2504] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:33 2022] DELETE /identity/v3/projects/3e1451a6c59a4ef0b0cfe3cc6b98c605 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:33.764058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363c57-baa7-4920-9712-23fe8773ea5d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:33.764333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363c57-baa7-4920-9712-23fe8773ea5d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:33.764609 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363c57-baa7-4920-9712-23fe8773ea5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:33.784738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3d363c57-baa7-4920-9712-23fe8773ea5d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:33.786014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3d363c57-baa7-4920-9712-23fe8773ea5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:33.855653 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1250/2505] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:33.864827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cdeea-b63e-4962-89b9-5749a4e0ab0a None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:33.865105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cdeea-b63e-4962-89b9-5749a4e0ab0a None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:33.865406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cdeea-b63e-4962-89b9-5749a4e0ab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:33.885415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d91cdeea-b63e-4962-89b9-5749a4e0ab0a None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:33.886689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d91cdeea-b63e-4962-89b9-5749a4e0ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:33.896665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1256/2506] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:33.901147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363c57-baa7-4920-9712-23fe8773ea5d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:33.912652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363c57-baa7-4920-9712-23fe8773ea5d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:33.918711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:33.919441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:33.919816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:33.920134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:33.920872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:33.921179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:33.923211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:33.935772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1251/2507] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:27:33.940359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cdeea-b63e-4962-89b9-5749a4e0ab0a None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:33.950419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cdeea-b63e-4962-89b9-5749a4e0ab0a None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:33.956212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859234ab-ff79-4609-99d6-c2cd0ae0e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:33.956896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859234ab-ff79-4609-99d6-c2cd0ae0e5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:33.957157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859234ab-ff79-4609-99d6-c2cd0ae0e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:33.957430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859234ab-ff79-4609-99d6-c2cd0ae0e5bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:33.958183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859234ab-ff79-4609-99d6-c2cd0ae0e5bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:33.960086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859234ab-ff79-4609-99d6-c2cd0ae0e5bd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:33.989579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1257/2508] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:33 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:33.993711 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:34.004386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f544a2-c990-41d0-81fc-60b5f1eb807c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:34.057786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:34.067344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19a47-831c-469f-9491-b429dfc7d05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:34.068269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19a47-831c-469f-9491-b429dfc7d05f admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:34.069017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19a47-831c-469f-9491-b429dfc7d05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:34.069310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19a47-831c-469f-9491-b429dfc7d05f admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:34.070118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19a47-831c-469f-9491-b429dfc7d05f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:34.073677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19a47-831c-469f-9491-b429dfc7d05f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:34.096648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1252/2509] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:33 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:34.101341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859234ab-ff79-4609-99d6-c2cd0ae0e5bd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:34.112728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859234ab-ff79-4609-99d6-c2cd0ae0e5bd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:34.117804 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fc65-5f26-460c-93c7-45ef0393c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:34.118479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fc65-5f26-460c-93c7-45ef0393c360 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:34.118765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fc65-5f26-460c-93c7-45ef0393c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:34.119049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fc65-5f26-460c-93c7-45ef0393c360 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:34.119867 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508fc65-5f26-460c-93c7-45ef0393c360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:34.121484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508fc65-5f26-460c-93c7-45ef0393c360 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:34.133129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1258/2510] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:34.137293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19a47-831c-469f-9491-b429dfc7d05f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:34.149418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19a47-831c-469f-9491-b429dfc7d05f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:34.155799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c38a3-0cb4-43ba-becb-9398a8731b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:34.156467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c38a3-0cb4-43ba-becb-9398a8731b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:34.156752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c38a3-0cb4-43ba-becb-9398a8731b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:34.157032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c38a3-0cb4-43ba-becb-9398a8731b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:34.157786 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c38a3-0cb4-43ba-becb-9398a8731b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:34.159420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c38a3-0cb4-43ba-becb-9398a8731b23 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:34.170505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1253/2511] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:34.176474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fc65-5f26-460c-93c7-45ef0393c360 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:34.191414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fc65-5f26-460c-93c7-45ef0393c360 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:34.198698 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:34.199994 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:34.200328 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:34.200732 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] PATH_INFO: `/v3/projects/d944af7f74394542ac7b6cb64beb6070/users/fc989f2fe18346fba53a08559e53d4b1/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:34.212084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:34.217412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d944af7f74394542ac7b6cb64beb6070, user_id=fc989f2fe18346fba53a08559e53d4b1, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:34.220492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:34.235790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1259/2512] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:34 2022] PUT /identity/v3/projects/d944af7f74394542ac7b6cb64beb6070/users/fc989f2fe18346fba53a08559e53d4b1/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:34.249608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef7559-d26d-4f98-bc2f-513c2a90422b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:34.250176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef7559-d26d-4f98-bc2f-513c2a90422b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:34.250176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef7559-d26d-4f98-bc2f-513c2a90422b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:34.283664 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-baef7559-d26d-4f98-bc2f-513c2a90422b None None] MFA Rules not processed for user `fc989f2fe18346fba53a08559e53d4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:34.284879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-baef7559-d26d-4f98-bc2f-513c2a90422b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:34.369534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1254/2513] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:34.381032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:34.384016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:34.447726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb627dc-697b-4842-b246-9d1b43b23105 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:34.452334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:34.452992 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:34.453277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:34.453552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:34.462491 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:34.463956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:34.474584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1260/2514] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:35.890289 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6736-8199-496d-a1ca-b0e1702cc819 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:35.890834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6736-8199-496d-a1ca-b0e1702cc819 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:35.890834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6736-8199-496d-a1ca-b0e1702cc819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:35.911559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-46dd6736-8199-496d-a1ca-b0e1702cc819 None None] MFA Rules not processed for user `fc989f2fe18346fba53a08559e53d4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:35.912802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-46dd6736-8199-496d-a1ca-b0e1702cc819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:35.921007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1255/2515] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:36.137625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:36.139192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:36.208999 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919030a7-7d87-49e0-a8a3-f5c38da997d3 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:36.213834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:36.214731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:36.215012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:36.215284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] PATH_INFO: `/v3/users/b5f2507bcde54d16826249975646b192` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:36.217236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f2507bcde54d16826249975646b192)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:36.219442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:36.274958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f2507bcde54d16826249975646b192'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.292655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f2507bcde54d16826249975646b192'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.311137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f2507bcde54d16826249975646b192'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.311963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f2507bcde54d16826249975646b192'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.312587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f2507bcde54d16826249975646b192'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.326862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1261/2516] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:36 2022] DELETE /identity/v3/users/b5f2507bcde54d16826249975646b192 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:36.331476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5d58a-0e54-4607-b040-3061b54f1c6e None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:36.331763 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5d58a-0e54-4607-b040-3061b54f1c6e None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:36.332028 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5d58a-0e54-4607-b040-3061b54f1c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:36.352422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2be5d58a-0e54-4607-b040-3061b54f1c6e None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:36.353644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2be5d58a-0e54-4607-b040-3061b54f1c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:36.427716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1256/2517] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:36.438920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:36.439626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:36.458617 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:36.521266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbba943-e51e-40ab-9738-8c81446a2195 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:36.530787 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:36.531460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:36.531810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:36.532011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:36.547357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:36.548901 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:36.560615 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1262/2518] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:36.881170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5d58a-0e54-4607-b040-3061b54f1c6e None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:36.882789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2be5d58a-0e54-4607-b040-3061b54f1c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:36.902980 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5d58a-0e54-4607-b040-3061b54f1c6e None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:36.906381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:36.908034 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:36.909202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:36.910295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:36.910641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:36.910832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] PATH_INFO: `/v3/projects/de1dc77abef544e0bf24a788fb899424` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:36.912087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1dc77abef544e0bf24a788fb899424)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:36.914658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:36.926349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46356aff-078b-4d85-88f8-3427646c6854 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:36.931316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:36.932224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:36.932529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:36.932796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] PATH_INFO: `/v3/users/88bf1f6867114bf585239802992e0e65` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:36.934893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=88bf1f6867114bf585239802992e0e65)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:36.936949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:36.968223 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1dc77abef544e0bf24a788fb899424', 'request_id': 'req-377da309-76e7-4d61-a533-5eed80802496'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.970155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:36.975213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1dc77abef544e0bf24a788fb899424', 'request_id': 'req-377da309-76e7-4d61-a533-5eed80802496'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.983864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1dc77abef544e0bf24a788fb899424', 'request_id': 'req-377da309-76e7-4d61-a533-5eed80802496'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.996726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] The token cache is being invalidate because project de1dc77abef544e0bf24a788fb899424 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:36.997183 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.997681 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:36.999418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1257/2519] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:36 2022] DELETE /identity/v3/projects/de1dc77abef544e0bf24a788fb899424 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:37.000543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:37.001964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.007308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88bf1f6867114bf585239802992e0e65'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.022124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88bf1f6867114bf585239802992e0e65'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.038401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88bf1f6867114bf585239802992e0e65'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.039624 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88bf1f6867114bf585239802992e0e65'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.040292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88bf1f6867114bf585239802992e0e65'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.057634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1263/2520] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:36 2022] DELETE /identity/v3/users/88bf1f6867114bf585239802992e0e65 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:37.057634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:37.057634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.089559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377da309-76e7-4d61-a533-5eed80802496 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:37.141378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a6389-6060-4a70-8351-a80cab1568dd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:37.147139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:37.148033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:37.148353 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:37.148591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] PATH_INFO: `/v3/users/b8b4f733aed944c7ad37cf4b4655d3e4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:37.149954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b4f733aed944c7ad37cf4b4655d3e4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:37.151929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:37.165536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0d5be-59aa-42ae-8856-8998f284ac70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:37.167032 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d5be-59aa-42ae-8856-8998f284ac70 service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:37.167370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d5be-59aa-42ae-8856-8998f284ac70 service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:37.167652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0d5be-59aa-42ae-8856-8998f284ac70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:37.170184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1cf0d5be-59aa-42ae-8856-8998f284ac70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.183983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0d5be-59aa-42ae-8856-8998f284ac70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:37.185830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0d5be-59aa-42ae-8856-8998f284ac70 service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:37.186569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:37.201719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1258/2521] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:37.203758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2ddc-5797-4425-bfa9-313a53184164 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:37.204039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2ddc-5797-4425-bfa9-313a53184164 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:37.204396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a2ddc-5797-4425-bfa9-313a53184164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:37.213524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b4f733aed944c7ad37cf4b4655d3e4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.225045 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-a97a2ddc-5797-4425-bfa9-313a53184164 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:37.226571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a97a2ddc-5797-4425-bfa9-313a53184164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.226714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b4f733aed944c7ad37cf4b4655d3e4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.240228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1259/2522] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:37.249056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b4f733aed944c7ad37cf4b4655d3e4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.250364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a2ddc-5797-4425-bfa9-313a53184164 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:37.251972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a97a2ddc-5797-4425-bfa9-313a53184164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.256519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b4f733aed944c7ad37cf4b4655d3e4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.257712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b4f733aed944c7ad37cf4b4655d3e4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.276020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1264/2523] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:37 2022] DELETE /identity/v3/users/b8b4f733aed944c7ad37cf4b4655d3e4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:37.343522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a2ddc-5797-4425-bfa9-313a53184164 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:37.374006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:37.416349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b0bc0-7ff3-4b52-ba01-9e9fbd4e767d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:37.417133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b0bc0-7ff3-4b52-ba01-9e9fbd4e767d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:37.417456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b0bc0-7ff3-4b52-ba01-9e9fbd4e767d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:37.417781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b0bc0-7ff3-4b52-ba01-9e9fbd4e767d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:37.420450 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-372b0bc0-7ff3-4b52-ba01-9e9fbd4e767d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.506862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b0bc0-7ff3-4b52-ba01-9e9fbd4e767d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:37.509130 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b0bc0-7ff3-4b52-ba01-9e9fbd4e767d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:37.520483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1260/2524] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:37.695238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:37.697088 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.712071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4cf29-69d1-4e28-8dd4-fc00c219e73c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:37.718554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:37.724561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:37.730040 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:37.730529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] PATH_INFO: `/v3/projects/72b0b142a5b14c0387aeec2d539be5d8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:37.733284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b0b142a5b14c0387aeec2d539be5d8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:37.735493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:37.790207 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b0b142a5b14c0387aeec2d539be5d8', 'request_id': 'req-7f151e44-674b-4e18-a481-6486cb931958'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.798435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b0b142a5b14c0387aeec2d539be5d8', 'request_id': 'req-7f151e44-674b-4e18-a481-6486cb931958'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.805008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b0b142a5b14c0387aeec2d539be5d8', 'request_id': 'req-7f151e44-674b-4e18-a481-6486cb931958'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.816196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] The token cache is being invalidate because project 72b0b142a5b14c0387aeec2d539be5d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:37.816479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.817220 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.818773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1265/2525] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:37 2022] DELETE /identity/v3/projects/72b0b142a5b14c0387aeec2d539be5d8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:37.823918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754901f5-0cdf-42b1-870e-4bdb2e89bfc1 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:37.824204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754901f5-0cdf-42b1-870e-4bdb2e89bfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:37.824502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754901f5-0cdf-42b1-870e-4bdb2e89bfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:37.843821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-754901f5-0cdf-42b1-870e-4bdb2e89bfc1 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:37.845076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-754901f5-0cdf-42b1-870e-4bdb2e89bfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.899918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:37.901647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.910428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1261/2526] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:37.917064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f151e44-674b-4e18-a481-6486cb931958 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:37.919659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03697f73-b589-40f7-a999-094862f5ccf9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:37.919958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03697f73-b589-40f7-a999-094862f5ccf9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:37.920293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03697f73-b589-40f7-a999-094862f5ccf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:37.923975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:37.925032 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:37.925309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:37.925537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] PATH_INFO: `/v3/projects/c9cdd7706ed243baa7033b27fbf046ea` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:37.926778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9cdd7706ed243baa7033b27fbf046ea)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:37.928899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:37.943273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-03697f73-b589-40f7-a999-094862f5ccf9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:37.944452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-03697f73-b589-40f7-a999-094862f5ccf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:37.954783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1262/2527] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:37.960339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03697f73-b589-40f7-a999-094862f5ccf9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:37.969765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03697f73-b589-40f7-a999-094862f5ccf9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:37.981931 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9cdd7706ed243baa7033b27fbf046ea', 'request_id': 'req-fe0ae797-3b84-42a4-a7a8-421fea4d1612'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.992165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9cdd7706ed243baa7033b27fbf046ea', 'request_id': 'req-fe0ae797-3b84-42a4-a7a8-421fea4d1612'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:37.999705 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9cdd7706ed243baa7033b27fbf046ea', 'request_id': 'req-fe0ae797-3b84-42a4-a7a8-421fea4d1612'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.010590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] The token cache is being invalidate because project c9cdd7706ed243baa7033b27fbf046ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:38.010878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.011553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.013887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1266/2528] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:37 2022] DELETE /identity/v3/projects/c9cdd7706ed243baa7033b27fbf046ea => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:38.017751 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.018896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.029986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.030658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.030935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.031204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.031944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:38.032236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.033171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:38.033918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.043105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:38.049836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1263/2529] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:27:38.051710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.053411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.065953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3f473-e5ab-45b6-a262-ad78cafda6ad admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.070577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5f27-363d-495b-a53f-8c9ca044eeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.071229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5f27-363d-495b-a53f-8c9ca044eeb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.071513 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5f27-363d-495b-a53f-8c9ca044eeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.071792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d5f27-363d-495b-a53f-8c9ca044eeb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.072476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d5f27-363d-495b-a53f-8c9ca044eeb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.074438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d5f27-363d-495b-a53f-8c9ca044eeb4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.088689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ae797-3b84-42a4-a7a8-421fea4d1612 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.094295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.095334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.095573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.095969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] PATH_INFO: `/v3/users/af83fbc7049e4340b05e09eea14ba516` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.097365 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af83fbc7049e4340b05e09eea14ba516)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.099381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.106729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1264/2530] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:38.107756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5f27-363d-495b-a53f-8c9ca044eeb4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.156792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af83fbc7049e4340b05e09eea14ba516'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.165196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af83fbc7049e4340b05e09eea14ba516'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.172920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d5f27-363d-495b-a53f-8c9ca044eeb4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.178188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e4e19-7456-4a15-9490-34c8fc12b768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.178372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af83fbc7049e4340b05e09eea14ba516'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.178867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e4e19-7456-4a15-9490-34c8fc12b768 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.179025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af83fbc7049e4340b05e09eea14ba516'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.179154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e4e19-7456-4a15-9490-34c8fc12b768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.179560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e4e19-7456-4a15-9490-34c8fc12b768 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.179657 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af83fbc7049e4340b05e09eea14ba516'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.180257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e4e19-7456-4a15-9490-34c8fc12b768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.182257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e4e19-7456-4a15-9490-34c8fc12b768 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.192030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1267/2531] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:38 2022] DELETE /identity/v3/users/af83fbc7049e4340b05e09eea14ba516 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:38.208522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1265/2532] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:38.209759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.211297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.282038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad76fa-d35a-4605-a00e-4adc70959182 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.287717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7f707-7842-4080-8067-7d67cd754d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.288589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7f707-7842-4080-8067-7d67cd754d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.288882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7f707-7842-4080-8067-7d67cd754d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.289162 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7f707-7842-4080-8067-7d67cd754d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.289927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7f707-7842-4080-8067-7d67cd754d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.291559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7f707-7842-4080-8067-7d67cd754d3d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.306307 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1268/2533] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:38.307650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e4e19-7456-4a15-9490-34c8fc12b768 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.317875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e4e19-7456-4a15-9490-34c8fc12b768 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.322631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6879723-00ff-4feb-9de4-5f9469dd66d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.323327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6879723-00ff-4feb-9de4-5f9469dd66d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.323605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6879723-00ff-4feb-9de4-5f9469dd66d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.323878 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6879723-00ff-4feb-9de4-5f9469dd66d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.324602 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6879723-00ff-4feb-9de4-5f9469dd66d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.326442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6879723-00ff-4feb-9de4-5f9469dd66d3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.337692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1266/2534] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:38.349762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7f707-7842-4080-8067-7d67cd754d3d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.365689 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7f707-7842-4080-8067-7d67cd754d3d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.371595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.372951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.373243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.373522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] PATH_INFO: `/v3/projects/3ded73fff49a4611acb1ef2937601fa4/users/ba84a806bd5a499d8c5dc5c81a91fb0e/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.390741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ded73fff49a4611acb1ef2937601fa4, user_id=ba84a806bd5a499d8c5dc5c81a91fb0e, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.393419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.411523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1269/2535] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:38 2022] PUT /identity/v3/projects/3ded73fff49a4611acb1ef2937601fa4/users/ba84a806bd5a499d8c5dc5c81a91fb0e/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:38.422774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd61db-2843-42aa-8173-2734b92b19a7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.423065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd61db-2843-42aa-8173-2734b92b19a7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.423349 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd61db-2843-42aa-8173-2734b92b19a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.452465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ebfd61db-2843-42aa-8173-2734b92b19a7 None None] MFA Rules not processed for user `ba84a806bd5a499d8c5dc5c81a91fb0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:38.453689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ebfd61db-2843-42aa-8173-2734b92b19a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.495790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.497963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.503285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:38.533075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1267/2536] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:38.537880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd61db-2843-42aa-8173-2734b92b19a7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.587141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:38.596163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb17c62-0137-42ec-9006-c2e351a0d625 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.601183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.602097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.602377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.602656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] PATH_INFO: `/v3/projects/e3a128add7aa422385671d8b869044e5` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.603847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a128add7aa422385671d8b869044e5)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.605835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.607013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd61db-2843-42aa-8173-2734b92b19a7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.612142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170967e-6380-44e9-8304-a662b3de0779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.612784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170967e-6380-44e9-8304-a662b3de0779 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.613095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170967e-6380-44e9-8304-a662b3de0779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.613373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170967e-6380-44e9-8304-a662b3de0779 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.614224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170967e-6380-44e9-8304-a662b3de0779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.616220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170967e-6380-44e9-8304-a662b3de0779 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.653566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1268/2537] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:38.657817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170967e-6380-44e9-8304-a662b3de0779 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.669713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a128add7aa422385671d8b869044e5', 'request_id': 'req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.680986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a128add7aa422385671d8b869044e5', 'request_id': 'req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.690165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a128add7aa422385671d8b869044e5', 'request_id': 'req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.702327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] The token cache is being invalidate because project e3a128add7aa422385671d8b869044e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:38.702849 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.703564 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3bd007b8-5e85-4d2a-9e17-df8dc3303eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:38.705600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1270/2538] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:38 2022] DELETE /identity/v3/projects/e3a128add7aa422385671d8b869044e5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:38.710547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69ab9-e96d-4480-b0cc-79b68254c3d8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.711052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69ab9-e96d-4480-b0cc-79b68254c3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.711479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f69ab9-e96d-4480-b0cc-79b68254c3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.731992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170967e-6380-44e9-8304-a662b3de0779 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.737871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.738542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.738826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.739108 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.739801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.741233 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-63f69ab9-e96d-4480-b0cc-79b68254c3d8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:38.741840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.742578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-63f69ab9-e96d-4480-b0cc-79b68254c3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.751536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1271/2539] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:38.761395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf22fd-e65d-4983-8efe-a32f3fed250b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.761717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf22fd-e65d-4983-8efe-a32f3fed250b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.762007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf22fd-e65d-4983-8efe-a32f3fed250b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.764052 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1269/2540] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:38.767924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.769777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.782624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9223f57-d124-40f4-ada8-4e849482f44c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.782828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-25cf22fd-e65d-4983-8efe-a32f3fed250b None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:38.783984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-25cf22fd-e65d-4983-8efe-a32f3fed250b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.788024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5663660-e045-4801-844e-18efc4c7c952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.789520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5663660-e045-4801-844e-18efc4c7c952 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.789944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5663660-e045-4801-844e-18efc4c7c952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.790127 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5663660-e045-4801-844e-18efc4c7c952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.790876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5663660-e045-4801-844e-18efc4c7c952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.792518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1272/2541] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:38.792671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5663660-e045-4801-844e-18efc4c7c952 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.797621 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf22fd-e65d-4983-8efe-a32f3fed250b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.808671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1270/2542] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:38.808858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf22fd-e65d-4983-8efe-a32f3fed250b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.810399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5663660-e045-4801-844e-18efc4c7c952 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.813569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.814292 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.814578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.814872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.815578 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:38.815878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.817618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.820604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5663660-e045-4801-844e-18efc4c7c952 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.825772 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81232c1d-d632-4717-917f-b42f48469d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.826636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81232c1d-d632-4717-917f-b42f48469d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.826910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81232c1d-d632-4717-917f-b42f48469d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.827207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81232c1d-d632-4717-917f-b42f48469d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.827937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81232c1d-d632-4717-917f-b42f48469d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.829574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81232c1d-d632-4717-917f-b42f48469d61 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.830374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1273/2543] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:27:38.835057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.842614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1271/2544] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:38.845005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d114976-d96e-4eaf-bc1c-ec29ab5df878 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.847207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81232c1d-d632-4717-917f-b42f48469d61 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.850304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eb56f-d774-4b7f-a73f-36fc0401002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.850970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eb56f-d774-4b7f-a73f-36fc0401002e admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.851251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eb56f-d774-4b7f-a73f-36fc0401002e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.851534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20eb56f-d774-4b7f-a73f-36fc0401002e admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.852231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20eb56f-d774-4b7f-a73f-36fc0401002e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.854181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20eb56f-d774-4b7f-a73f-36fc0401002e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.860967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81232c1d-d632-4717-917f-b42f48469d61 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:38.866509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bfbda-dd67-4fcb-860e-983e19c81f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:38.867428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfbda-dd67-4fcb-860e-983e19c81f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.867703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfbda-dd67-4fcb-860e-983e19c81f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.868007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bfbda-dd67-4fcb-860e-983e19c81f76 admin admin] PATH_INFO: `/v3/projects/c4a755a68e9f4e98955e05170c854d89/users/3e10d2cd060a43bcb56e5d0421203644/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.884363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bfbda-dd67-4fcb-860e-983e19c81f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a755a68e9f4e98955e05170c854d89, user_id=3e10d2cd060a43bcb56e5d0421203644, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:38.884628 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1274/2545] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:38.888569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bfbda-dd67-4fcb-860e-983e19c81f76 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:38.889529 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eb56f-d774-4b7f-a73f-36fc0401002e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:38.903856 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1272/2546] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:38 2022] PUT /identity/v3/projects/c4a755a68e9f4e98955e05170c854d89/users/3e10d2cd060a43bcb56e5d0421203644/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:38.915503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e722e3-3033-4f85-bcf6-8884f0940064 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:38.916147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e722e3-3033-4f85-bcf6-8884f0940064 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:38.916460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e722e3-3033-4f85-bcf6-8884f0940064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:38.938400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:38.944454 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-41e722e3-3033-4f85-bcf6-8884f0940064 None None] MFA Rules not processed for user `3e10d2cd060a43bcb56e5d0421203644`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:38.945933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-41e722e3-3033-4f85-bcf6-8884f0940064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:38.957206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20eb56f-d774-4b7f-a73f-36fc0401002e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.012124 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1273/2547] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:39.015242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63ade7-2990-4172-bc54-6d257122f863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.015932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63ade7-2990-4172-bc54-6d257122f863 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.016218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63ade7-2990-4172-bc54-6d257122f863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.016502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63ade7-2990-4172-bc54-6d257122f863 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.016983 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e722e3-3033-4f85-bcf6-8884f0940064 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.017180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63ade7-2990-4172-bc54-6d257122f863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.019191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63ade7-2990-4172-bc54-6d257122f863 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.024281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:39.029113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e722e3-3033-4f85-bcf6-8884f0940064 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.034818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3981b-cd79-405f-8f3a-79d670e128c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.035766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3981b-cd79-405f-8f3a-79d670e128c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.036471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3981b-cd79-405f-8f3a-79d670e128c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.036715 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3981b-cd79-405f-8f3a-79d670e128c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.037458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3981b-cd79-405f-8f3a-79d670e128c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.039485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb3981b-cd79-405f-8f3a-79d670e128c6 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.044611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1275/2548] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:38 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:39.048060 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63ade7-2990-4172-bc54-6d257122f863 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.058440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63ade7-2990-4172-bc54-6d257122f863 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.063676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35944d-f918-43e9-8c7a-92aac38bd0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.064354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35944d-f918-43e9-8c7a-92aac38bd0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.064640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35944d-f918-43e9-8c7a-92aac38bd0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.064967 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35944d-f918-43e9-8c7a-92aac38bd0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.065674 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35944d-f918-43e9-8c7a-92aac38bd0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.067123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1274/2549] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:39 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:39.067334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35944d-f918-43e9-8c7a-92aac38bd0b3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.071263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3981b-cd79-405f-8f3a-79d670e128c6 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.078706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1276/2550] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:39.082736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35944d-f918-43e9-8c7a-92aac38bd0b3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.093030 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35944d-f918-43e9-8c7a-92aac38bd0b3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.140487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3981b-cd79-405f-8f3a-79d670e128c6 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.145636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511202d-7af0-46f8-a378-6a21cd6bc680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.146332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511202d-7af0-46f8-a378-6a21cd6bc680 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.146762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511202d-7af0-46f8-a378-6a21cd6bc680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.147042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511202d-7af0-46f8-a378-6a21cd6bc680 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.147748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511202d-7af0-46f8-a378-6a21cd6bc680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.149746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511202d-7af0-46f8-a378-6a21cd6bc680 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.153395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db29c2-9e1d-4b56-b6aa-805d867601f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.154114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db29c2-9e1d-4b56-b6aa-805d867601f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.154399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db29c2-9e1d-4b56-b6aa-805d867601f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.154677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db29c2-9e1d-4b56-b6aa-805d867601f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.155403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db29c2-9e1d-4b56-b6aa-805d867601f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.156984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db29c2-9e1d-4b56-b6aa-805d867601f3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.171039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1277/2551] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:39.171728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1275/2552] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:39 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:39.172199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db29c2-9e1d-4b56-b6aa-805d867601f3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.175443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511202d-7af0-46f8-a378-6a21cd6bc680 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.182431 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db29c2-9e1d-4b56-b6aa-805d867601f3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.185432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511202d-7af0-46f8-a378-6a21cd6bc680 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.187685 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dd191-c9e0-41f1-9915-cc7701e593ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.188561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd191-c9e0-41f1-9915-cc7701e593ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.188847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd191-c9e0-41f1-9915-cc7701e593ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.189092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dd191-c9e0-41f1-9915-cc7701e593ce admin admin] PATH_INFO: `/v3/projects/6250df92cfca4314a656ceb44f472fd4/users/2a50ccb505c44e4eafccec7d6ea5a22c/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.190477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97870b30-58ae-43d3-b823-b046593b2b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.191134 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97870b30-58ae-43d3-b823-b046593b2b00 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.191629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97870b30-58ae-43d3-b823-b046593b2b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.191903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97870b30-58ae-43d3-b823-b046593b2b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.193505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97870b30-58ae-43d3-b823-b046593b2b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.195311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97870b30-58ae-43d3-b823-b046593b2b00 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.204461 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd191-c9e0-41f1-9915-cc7701e593ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6250df92cfca4314a656ceb44f472fd4, user_id=2a50ccb505c44e4eafccec7d6ea5a22c, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.206975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1276/2553] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:39.208400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dd191-c9e0-41f1-9915-cc7701e593ce admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.211736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97870b30-58ae-43d3-b823-b046593b2b00 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.222530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97870b30-58ae-43d3-b823-b046593b2b00 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.223287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1278/2554] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:39 2022] PUT /identity/v3/projects/6250df92cfca4314a656ceb44f472fd4/users/2a50ccb505c44e4eafccec7d6ea5a22c/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:39.234900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6ec9c-956c-4398-812d-5ac62267a0d0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.235227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6ec9c-956c-4398-812d-5ac62267a0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.235514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6ec9c-956c-4398-812d-5ac62267a0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.264487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-33c6ec9c-956c-4398-812d-5ac62267a0d0 None None] MFA Rules not processed for user `2a50ccb505c44e4eafccec7d6ea5a22c`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:39.265785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-33c6ec9c-956c-4398-812d-5ac62267a0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:39.291309 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ffab9-a5f4-489a-90c0-03b05c1f9e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.292202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ffab9-a5f4-489a-90c0-03b05c1f9e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.292476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ffab9-a5f4-489a-90c0-03b05c1f9e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.292752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ffab9-a5f4-489a-90c0-03b05c1f9e60 admin admin] PATH_INFO: `/v3/projects/ef50e15f9cf14b8bb72ea9520aa0dfa3/users/5ec5063b921f407582197671a3cce6a8/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.298079 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:39.313308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ffab9-a5f4-489a-90c0-03b05c1f9e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef50e15f9cf14b8bb72ea9520aa0dfa3, user_id=5ec5063b921f407582197671a3cce6a8, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.316047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ffab9-a5f4-489a-90c0-03b05c1f9e60 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.331903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1277/2555] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:39 2022] PUT /identity/v3/projects/ef50e15f9cf14b8bb72ea9520aa0dfa3/users/5ec5063b921f407582197671a3cce6a8/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:39.335903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ffab9-a5f4-489a-90c0-03b05c1f9e60 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.342418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1279/2556] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:39.346788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6ec9c-956c-4398-812d-5ac62267a0d0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.358855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6ec9c-956c-4398-812d-5ac62267a0d0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.394906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:39.413388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ffab9-a5f4-489a-90c0-03b05c1f9e60 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.420117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217bcb4-5f6d-477c-b4d5-48665ddf4962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.420952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217bcb4-5f6d-477c-b4d5-48665ddf4962 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.421237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217bcb4-5f6d-477c-b4d5-48665ddf4962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.421514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217bcb4-5f6d-477c-b4d5-48665ddf4962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.422302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217bcb4-5f6d-477c-b4d5-48665ddf4962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.423942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217bcb4-5f6d-477c-b4d5-48665ddf4962 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.428316 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb41bb-0e27-48c1-a30f-e87e6f7b195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.429007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb41bb-0e27-48c1-a30f-e87e6f7b195a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.429293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb41bb-0e27-48c1-a30f-e87e6f7b195a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.429595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb41bb-0e27-48c1-a30f-e87e6f7b195a admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.430319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb41bb-0e27-48c1-a30f-e87e6f7b195a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.432379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb41bb-0e27-48c1-a30f-e87e6f7b195a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.435201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1278/2557] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:39.439343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217bcb4-5f6d-477c-b4d5-48665ddf4962 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.450598 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217bcb4-5f6d-477c-b4d5-48665ddf4962 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.455982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe005c-140a-4241-a46f-8d94d7b4fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.456883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe005c-140a-4241-a46f-8d94d7b4fd9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.457167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe005c-140a-4241-a46f-8d94d7b4fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.457446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe005c-140a-4241-a46f-8d94d7b4fd9a admin admin] PATH_INFO: `/v3/projects/ef50e15f9cf14b8bb72ea9520aa0dfa3/users/5ec5063b921f407582197671a3cce6a8/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.459552 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe005c-140a-4241-a46f-8d94d7b4fd9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef50e15f9cf14b8bb72ea9520aa0dfa3, user_id=5ec5063b921f407582197671a3cce6a8, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.460077 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1280/2558] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:39 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:39.462234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe005c-140a-4241-a46f-8d94d7b4fd9a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.467130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb41bb-0e27-48c1-a30f-e87e6f7b195a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.475722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1279/2559] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:39 2022] PUT /identity/v3/projects/ef50e15f9cf14b8bb72ea9520aa0dfa3/users/5ec5063b921f407582197671a3cce6a8/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:39.486538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dab5c-3876-4d39-b9d8-06023d0abcb7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.486837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dab5c-3876-4d39-b9d8-06023d0abcb7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.487118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64dab5c-3876-4d39-b9d8-06023d0abcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.516535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f64dab5c-3876-4d39-b9d8-06023d0abcb7 None None] MFA Rules not processed for user `5ec5063b921f407582197671a3cce6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:39.517831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f64dab5c-3876-4d39-b9d8-06023d0abcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:39.533029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb41bb-0e27-48c1-a30f-e87e6f7b195a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.588395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1280/2560] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:39.592111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdbb88-a869-42d8-9f03-cb79ab876dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.592800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbb88-a869-42d8-9f03-cb79ab876dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.593082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbb88-a869-42d8-9f03-cb79ab876dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.593355 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbb88-a869-42d8-9f03-cb79ab876dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.594152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdbb88-a869-42d8-9f03-cb79ab876dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.596136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdbb88-a869-42d8-9f03-cb79ab876dd3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.600132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dab5c-3876-4d39-b9d8-06023d0abcb7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.601540 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f64dab5c-3876-4d39-b9d8-06023d0abcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:39.604987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:39.619023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1281/2561] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:39 2022] POST /identity/v3/users => generated 300 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:39.623193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdbb88-a869-42d8-9f03-cb79ab876dd3 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.634301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdbb88-a869-42d8-9f03-cb79ab876dd3 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.640086 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f59ff-a702-4ad1-9ec0-1ef805a2ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.640947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f59ff-a702-4ad1-9ec0-1ef805a2ad84 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.641208 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f59ff-a702-4ad1-9ec0-1ef805a2ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.641489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f59ff-a702-4ad1-9ec0-1ef805a2ad84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.642302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f59ff-a702-4ad1-9ec0-1ef805a2ad84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.643892 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f59ff-a702-4ad1-9ec0-1ef805a2ad84 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.657953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1282/2562] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:39.659309 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f59ff-a702-4ad1-9ec0-1ef805a2ad84 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.665816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64dab5c-3876-4d39-b9d8-06023d0abcb7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.669869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f59ff-a702-4ad1-9ec0-1ef805a2ad84 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.670016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.670641 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.670919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.671206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.673133 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:39.674712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad221d-0bd5-4e43-9a09-331175ebaaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.675604 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad221d-0bd5-4e43-9a09-331175ebaaaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.675887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad221d-0bd5-4e43-9a09-331175ebaaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.676168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad221d-0bd5-4e43-9a09-331175ebaaaf admin admin] PATH_INFO: `/v3/projects/37d7e1c5e37b4fe69c8c1ee0f4377133/users/c6edc8838d8943f0839c9e3c1dce004f/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.694041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad221d-0bd5-4e43-9a09-331175ebaaaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d7e1c5e37b4fe69c8c1ee0f4377133, user_id=c6edc8838d8943f0839c9e3c1dce004f, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.696797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad221d-0bd5-4e43-9a09-331175ebaaaf admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.711939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1283/2563] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:39 2022] PUT /identity/v3/projects/37d7e1c5e37b4fe69c8c1ee0f4377133/users/c6edc8838d8943f0839c9e3c1dce004f/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:39.715824 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad221d-0bd5-4e43-9a09-331175ebaaaf admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.736479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.738058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.748118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1281/2564] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:27:39.788947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad221d-0bd5-4e43-9a09-331175ebaaaf admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.794054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cdfe8-d15b-4a94-8065-807cfcdb18ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.794744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cdfe8-d15b-4a94-8065-807cfcdb18ea admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.795023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cdfe8-d15b-4a94-8065-807cfcdb18ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.795298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cdfe8-d15b-4a94-8065-807cfcdb18ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.796037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cdfe8-d15b-4a94-8065-807cfcdb18ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.797688 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cdfe8-d15b-4a94-8065-807cfcdb18ea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.815009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1284/2565] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:39.816291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.827644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab092265-4e94-4205-989b-7352aa57a7ad service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:39.832637 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:39.842921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.842921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.842921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] PATH_INFO: `/v3/projects/37d7e1c5e37b4fe69c8c1ee0f4377133/users/c6edc8838d8943f0839c9e3c1dce004f/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.842921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37d7e1c5e37b4fe69c8c1ee0f4377133, user_id=c6edc8838d8943f0839c9e3c1dce004f, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:39.842921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:39.843679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:39.852282 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1282/2566] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:39 2022] PUT /identity/v3/projects/37d7e1c5e37b4fe69c8c1ee0f4377133/users/c6edc8838d8943f0839c9e3c1dce004f/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:39.863791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e096d-011a-4287-af77-a7e6ea2638c3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:39.864080 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e096d-011a-4287-af77-a7e6ea2638c3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:39.864397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e096d-011a-4287-af77-a7e6ea2638c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:39.897819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-3d4e096d-011a-4287-af77-a7e6ea2638c3 None None] MFA Rules not processed for user `c6edc8838d8943f0839c9e3c1dce004f`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:39.899048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3d4e096d-011a-4287-af77-a7e6ea2638c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:39.974453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1285/2567] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:39.989193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:39.991734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:40.068280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9018116-076d-4216-b44c-bb60f5701390 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:40.072504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef389da-9c3a-4954-9b8a-d11ac611e11f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:40.073161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef389da-9c3a-4954-9b8a-d11ac611e11f service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:40.073503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef389da-9c3a-4954-9b8a-d11ac611e11f service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:40.073908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef389da-9c3a-4954-9b8a-d11ac611e11f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:40.084590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef389da-9c3a-4954-9b8a-d11ac611e11f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:40.086076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef389da-9c3a-4954-9b8a-d11ac611e11f service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:40.100903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1283/2568] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:40.130798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e096d-011a-4287-af77-a7e6ea2638c3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:40.176878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:40.200417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e096d-011a-4287-af77-a7e6ea2638c3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:40.205375 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:40.206349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:40.206630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:40.207065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] PATH_INFO: `/v3/users/2a50ccb505c44e4eafccec7d6ea5a22c` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:40.208416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a50ccb505c44e4eafccec7d6ea5a22c)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:40.210434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:40.267289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a50ccb505c44e4eafccec7d6ea5a22c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:40.276371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a50ccb505c44e4eafccec7d6ea5a22c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:40.289227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a50ccb505c44e4eafccec7d6ea5a22c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:40.290105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a50ccb505c44e4eafccec7d6ea5a22c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:40.290708 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a50ccb505c44e4eafccec7d6ea5a22c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:40.303881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1286/2569] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:40 2022] DELETE /identity/v3/users/2a50ccb505c44e4eafccec7d6ea5a22c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:40.314904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2dbd6-32ba-40c1-89e6-2d59792346ce None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:40.315160 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2dbd6-32ba-40c1-89e6-2d59792346ce None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:40.315462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa2dbd6-32ba-40c1-89e6-2d59792346ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:40.334940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5fa2dbd6-32ba-40c1-89e6-2d59792346ce None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:40.336182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5fa2dbd6-32ba-40c1-89e6-2d59792346ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:40.438128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1284/2570] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:40.450877 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:40.452489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:40.522672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729b2f2-b12d-445c-b6dc-198aac5e1c3e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:40.526789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992e453-542c-4d14-b8ca-948c31df995f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:40.527518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e453-542c-4d14-b8ca-948c31df995f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:40.528480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e453-542c-4d14-b8ca-948c31df995f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:40.528480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e453-542c-4d14-b8ca-948c31df995f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:40.540102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992e453-542c-4d14-b8ca-948c31df995f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:40.541801 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e992e453-542c-4d14-b8ca-948c31df995f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:40.554187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1287/2571] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:40.867180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2dbd6-32ba-40c1-89e6-2d59792346ce None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:40.868627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5fa2dbd6-32ba-40c1-89e6-2d59792346ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:40.885109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa2dbd6-32ba-40c1-89e6-2d59792346ce None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:40.904977 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:40.906463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:40.907376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:40.908281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] PATH_INFO: `/v3/projects/6250df92cfca4314a656ceb44f472fd4` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:40.910143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6250df92cfca4314a656ceb44f472fd4)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:40.912538 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:40.977950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6250df92cfca4314a656ceb44f472fd4', 'request_id': 'req-ce0ffa74-f62d-44d4-b241-0dbf01c55702'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:40.984805 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6250df92cfca4314a656ceb44f472fd4', 'request_id': 'req-ce0ffa74-f62d-44d4-b241-0dbf01c55702'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:40.993800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6250df92cfca4314a656ceb44f472fd4', 'request_id': 'req-ce0ffa74-f62d-44d4-b241-0dbf01c55702'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.005002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] The token cache is being invalidate because project 6250df92cfca4314a656ceb44f472fd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:41.005338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.006056 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.007689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1285/2572] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:40 2022] DELETE /identity/v3/projects/6250df92cfca4314a656ceb44f472fd4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:41.011835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992e453-542c-4d14-b8ca-948c31df995f service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:41.022485 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992e453-542c-4d14-b8ca-948c31df995f service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:41.092714 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:41.093212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:41.093502 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:41.093815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] PATH_INFO: `/v3/users/c6edc8838d8943f0839c9e3c1dce004f` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:41.095216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6edc8838d8943f0839c9e3c1dce004f)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:41.097277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:41.100568 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:41.164201 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6edc8838d8943f0839c9e3c1dce004f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.173996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6edc8838d8943f0839c9e3c1dce004f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.191437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6edc8838d8943f0839c9e3c1dce004f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.192635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6edc8838d8943f0839c9e3c1dce004f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.193314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6edc8838d8943f0839c9e3c1dce004f'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.214429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1288/2573] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:41 2022] DELETE /identity/v3/users/c6edc8838d8943f0839c9e3c1dce004f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:41.318993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:41.320704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:41.347615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:41.400827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ffa74-f62d-44d4-b241-0dbf01c55702 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:41.458993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b8268-bab8-466c-805d-009b0dd4d87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:41.459747 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8268-bab8-466c-805d-009b0dd4d87f service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:41.460149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8268-bab8-466c-805d-009b0dd4d87f service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:41.460345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b8268-bab8-466c-805d-009b0dd4d87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:41.466958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-526b8268-bab8-466c-805d-009b0dd4d87f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:41.546923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b8268-bab8-466c-805d-009b0dd4d87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:41.549321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b8268-bab8-466c-805d-009b0dd4d87f service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:41.566376 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1286/2574] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:41.701191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:41.702672 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:41.793499 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893af14-0368-42e9-831c-c34e726d25f4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:41.798882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:41.799770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:41.800189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:41.800474 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] PATH_INFO: `/v3/projects/37d7e1c5e37b4fe69c8c1ee0f4377133` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:41.801764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37d7e1c5e37b4fe69c8c1ee0f4377133)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:41.803809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:41.866764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37d7e1c5e37b4fe69c8c1ee0f4377133', 'request_id': 'req-633ad92f-d117-40af-851a-4cf8708b9994'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.882514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37d7e1c5e37b4fe69c8c1ee0f4377133', 'request_id': 'req-633ad92f-d117-40af-851a-4cf8708b9994'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.892074 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37d7e1c5e37b4fe69c8c1ee0f4377133', 'request_id': 'req-633ad92f-d117-40af-851a-4cf8708b9994'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.911673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] The token cache is being invalidate because project 37d7e1c5e37b4fe69c8c1ee0f4377133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:41.911985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.912861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-633ad92f-d117-40af-851a-4cf8708b9994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:41.917774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1289/2575] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:41 2022] DELETE /identity/v3/projects/37d7e1c5e37b4fe69c8c1ee0f4377133 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:41.919840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6f2dc-3656-4a55-bf5f-1585dc61f6dc None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:41.920316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6f2dc-3656-4a55-bf5f-1585dc61f6dc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:41.920613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6f2dc-3656-4a55-bf5f-1585dc61f6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:41.949211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-74e6f2dc-3656-4a55-bf5f-1585dc61f6dc None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:41.952564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-74e6f2dc-3656-4a55-bf5f-1585dc61f6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:42.067142 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1287/2576] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:42.076997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b68fa-8b32-4ab3-bf67-689e7b25fe2f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.077229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b68fa-8b32-4ab3-bf67-689e7b25fe2f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.077553 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b68fa-8b32-4ab3-bf67-689e7b25fe2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.112175 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7a0b68fa-8b32-4ab3-bf67-689e7b25fe2f None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:42.113772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7a0b68fa-8b32-4ab3-bf67-689e7b25fe2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:42.130127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1290/2577] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:42.134532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6f2dc-3656-4a55-bf5f-1585dc61f6dc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.146672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6f2dc-3656-4a55-bf5f-1585dc61f6dc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.152468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.153802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.154235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.154652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.155974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:42.156405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.158484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.192354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1288/2578] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:27:42.195209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b68fa-8b32-4ab3-bf67-689e7b25fe2f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.203785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:42.223573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b68fa-8b32-4ab3-bf67-689e7b25fe2f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.237855 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.239224 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.239511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.240283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.241173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.243280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.277802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1291/2579] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:42 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:42.281799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.291882 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa8fe8-7513-497b-a785-b17387a7ca84 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.309972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:42.344298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.346922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:42.365934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004113f-e1b4-4153-93f6-03deb1612f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.366645 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004113f-e1b4-4153-93f6-03deb1612f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.366937 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004113f-e1b4-4153-93f6-03deb1612f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.367265 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004113f-e1b4-4153-93f6-03deb1612f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.367974 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004113f-e1b4-4153-93f6-03deb1612f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.370047 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004113f-e1b4-4153-93f6-03deb1612f5c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.398644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1289/2580] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:42 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:42.400238 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004113f-e1b4-4153-93f6-03deb1612f5c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.420509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004113f-e1b4-4153-93f6-03deb1612f5c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.431967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ef133-49a5-49c9-980c-49bd79134d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.433232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef133-49a5-49c9-980c-49bd79134d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.433838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef133-49a5-49c9-980c-49bd79134d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.434759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ef133-49a5-49c9-980c-49bd79134d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.435682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ef133-49a5-49c9-980c-49bd79134d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.437634 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ef133-49a5-49c9-980c-49bd79134d39 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.438432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e348d-55f0-47d8-83ea-24232a205910 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.448352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d611c-c600-41d6-903e-bb873a0ed49f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.449288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d611c-c600-41d6-903e-bb873a0ed49f service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.449796 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d611c-c600-41d6-903e-bb873a0ed49f service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.450923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d611c-c600-41d6-903e-bb873a0ed49f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.454141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0f4d611c-c600-41d6-903e-bb873a0ed49f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:42.461844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1290/2581] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:42.465861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ef133-49a5-49c9-980c-49bd79134d39 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.479414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ef133-49a5-49c9-980c-49bd79134d39 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.486569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b232b2-3cce-4862-a749-40fc023004da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.487583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b232b2-3cce-4862-a749-40fc023004da admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.489099 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b232b2-3cce-4862-a749-40fc023004da admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.489555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b232b2-3cce-4862-a749-40fc023004da admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.490500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b232b2-3cce-4862-a749-40fc023004da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.492327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b232b2-3cce-4862-a749-40fc023004da admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.506836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1291/2582] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:42.511206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b232b2-3cce-4862-a749-40fc023004da admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.523862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b232b2-3cce-4862-a749-40fc023004da admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.531625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da8684-d46e-4002-b347-751849c92a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.532654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da8684-d46e-4002-b347-751849c92a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.532899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da8684-d46e-4002-b347-751849c92a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.533177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da8684-d46e-4002-b347-751849c92a58 admin admin] PATH_INFO: `/v3/projects/02c6777eea5b4606a88ec18223b6a6d0/users/0aeedf802e544510a08ca1507f72d3b4/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.551493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da8684-d46e-4002-b347-751849c92a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c6777eea5b4606a88ec18223b6a6d0, user_id=0aeedf802e544510a08ca1507f72d3b4, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.551712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d611c-c600-41d6-903e-bb873a0ed49f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.553199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d611c-c600-41d6-903e-bb873a0ed49f service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.554955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da8684-d46e-4002-b347-751849c92a58 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.571007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1292/2583] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:27:42.572381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1292/2584] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:42 2022] PUT /identity/v3/projects/02c6777eea5b4606a88ec18223b6a6d0/users/0aeedf802e544510a08ca1507f72d3b4/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:42.585516 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05e85b-04ec-4e74-9d47-db22558d5aba None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.585894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05e85b-04ec-4e74-9d47-db22558d5aba None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.586280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05e85b-04ec-4e74-9d47-db22558d5aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.630005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-cb05e85b-04ec-4e74-9d47-db22558d5aba None None] MFA Rules not processed for user `0aeedf802e544510a08ca1507f72d3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:42.631634 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cb05e85b-04ec-4e74-9d47-db22558d5aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:42.739549 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1293/2585] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:42.746921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da8684-d46e-4002-b347-751849c92a58 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.879760 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da8684-d46e-4002-b347-751849c92a58 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.893341 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b8023-7e0e-4572-8af9-2c37d8848c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:42.894140 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8023-7e0e-4572-8af9-2c37d8848c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:42.894478 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8023-7e0e-4572-8af9-2c37d8848c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:42.894706 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8023-7e0e-4572-8af9-2c37d8848c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:42.895548 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b8023-7e0e-4572-8af9-2c37d8848c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:42.897759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b8023-7e0e-4572-8af9-2c37d8848c58 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:42.933484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1293/2586] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:42 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:42.938071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05e85b-04ec-4e74-9d47-db22558d5aba None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:42.950959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05e85b-04ec-4e74-9d47-db22558d5aba None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:42.975636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:43.031665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1cdcc-cba0-405c-b3da-9481a3890a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:43.032440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1cdcc-cba0-405c-b3da-9481a3890a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.032788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1cdcc-cba0-405c-b3da-9481a3890a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.033676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1cdcc-cba0-405c-b3da-9481a3890a53 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.034489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1cdcc-cba0-405c-b3da-9481a3890a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:43.036523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1cdcc-cba0-405c-b3da-9481a3890a53 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:43.061333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1294/2587] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:42 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:43.065703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b8023-7e0e-4572-8af9-2c37d8848c58 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:43.080101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b8023-7e0e-4572-8af9-2c37d8848c58 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:43.086828 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11691bcd-cc43-4cac-bebf-a00428bcabba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:43.087420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691bcd-cc43-4cac-bebf-a00428bcabba admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.087729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691bcd-cc43-4cac-bebf-a00428bcabba admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.088005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11691bcd-cc43-4cac-bebf-a00428bcabba admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.088731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11691bcd-cc43-4cac-bebf-a00428bcabba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:43.090447 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11691bcd-cc43-4cac-bebf-a00428bcabba admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:43.109531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1294/2588] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:43.113772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1cdcc-cba0-405c-b3da-9481a3890a53 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:43.127008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1cdcc-cba0-405c-b3da-9481a3890a53 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:43.132607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fc843-02c5-48c2-a5eb-152fa70de5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:43.133545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fc843-02c5-48c2-a5eb-152fa70de5bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.133881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fc843-02c5-48c2-a5eb-152fa70de5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.134160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7fc843-02c5-48c2-a5eb-152fa70de5bc admin admin] PATH_INFO: `/v3/projects/255b3a59ca174b668c62b4c8a3400c74/users/ca9c046b41584527b2dffa1ddc5d2a70/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.153270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fc843-02c5-48c2-a5eb-152fa70de5bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=255b3a59ca174b668c62b4c8a3400c74, user_id=ca9c046b41584527b2dffa1ddc5d2a70, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:43.156214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7fc843-02c5-48c2-a5eb-152fa70de5bc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:43.170662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1295/2589] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:43 2022] PUT /identity/v3/projects/255b3a59ca174b668c62b4c8a3400c74/users/ca9c046b41584527b2dffa1ddc5d2a70/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:43.174693 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11691bcd-cc43-4cac-bebf-a00428bcabba admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:43.184969 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11691bcd-cc43-4cac-bebf-a00428bcabba admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:43.220294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:43.250136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7c628-bb3d-4fb6-b727-c6cd94ed7ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:43.250826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c628-bb3d-4fb6-b727-c6cd94ed7ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.251113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c628-bb3d-4fb6-b727-c6cd94ed7ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.251396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c628-bb3d-4fb6-b727-c6cd94ed7ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.252132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7c628-bb3d-4fb6-b727-c6cd94ed7ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:43.254017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7c628-bb3d-4fb6-b727-c6cd94ed7ce1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:43.268593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1295/2590] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:43.269884 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fc843-02c5-48c2-a5eb-152fa70de5bc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:43.281958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7fc843-02c5-48c2-a5eb-152fa70de5bc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:43.287145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:43.288253 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.288706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.288979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] PATH_INFO: `/v3/projects/255b3a59ca174b668c62b4c8a3400c74/users/ca9c046b41584527b2dffa1ddc5d2a70/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.291061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=255b3a59ca174b668c62b4c8a3400c74, user_id=ca9c046b41584527b2dffa1ddc5d2a70, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:43.293926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:43.306812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1296/2591] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:43 2022] PUT /identity/v3/projects/255b3a59ca174b668c62b4c8a3400c74/users/ca9c046b41584527b2dffa1ddc5d2a70/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:43.318455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb160dc-977f-4514-a9e1-a869ac34d5c8 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.318748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb160dc-977f-4514-a9e1-a869ac34d5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.319024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb160dc-977f-4514-a9e1-a869ac34d5c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.349409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-afb160dc-977f-4514-a9e1-a869ac34d5c8 None None] MFA Rules not processed for user `ca9c046b41584527b2dffa1ddc5d2a70`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:43.350703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-afb160dc-977f-4514-a9e1-a869ac34d5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:43.426516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1296/2592] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:43.439125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:43.440979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:43.502395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0171859-54da-4a44-b537-f92f0ac94634 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:43.506133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907bff73-6029-4493-bf6a-98150034832f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:43.506784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bff73-6029-4493-bf6a-98150034832f service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.507064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bff73-6029-4493-bf6a-98150034832f service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.507400 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bff73-6029-4493-bf6a-98150034832f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.517907 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bff73-6029-4493-bf6a-98150034832f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:43.519341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907bff73-6029-4493-bf6a-98150034832f service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:43.529669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1297/2593] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:43.604489 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb160dc-977f-4514-a9e1-a869ac34d5c8 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:43.669406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb160dc-977f-4514-a9e1-a869ac34d5c8 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:43.674020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:43.674902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.675184 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.675462 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] PATH_INFO: `/v3/users/0aeedf802e544510a08ca1507f72d3b4` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.676664 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aeedf802e544510a08ca1507f72d3b4)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:43.678657 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:43.735613 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aeedf802e544510a08ca1507f72d3b4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:43.743799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aeedf802e544510a08ca1507f72d3b4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:43.752377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:43.761738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aeedf802e544510a08ca1507f72d3b4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:43.762560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aeedf802e544510a08ca1507f72d3b4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:43.763296 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aeedf802e544510a08ca1507f72d3b4'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:43.775994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1297/2594] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:43 2022] DELETE /identity/v3/users/0aeedf802e544510a08ca1507f72d3b4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:43.781047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5313d9-f593-4790-b4aa-bd25c60108ba None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.781342 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5313d9-f593-4790-b4aa-bd25c60108ba None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.781643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5313d9-f593-4790-b4aa-bd25c60108ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.800138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-fa5313d9-f593-4790-b4aa-bd25c60108ba None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:43.801352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fa5313d9-f593-4790-b4aa-bd25c60108ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:43.866383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1298/2595] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:43.877226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:43.878856 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:43.948749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf730d-bc90-469a-97f9-6b7ca0add2e7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:43.953149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84bba6-a815-457a-b0c4-49bb081a6c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:43.954035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84bba6-a815-457a-b0c4-49bb081a6c56 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:43.954359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84bba6-a815-457a-b0c4-49bb081a6c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:43.954635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84bba6-a815-457a-b0c4-49bb081a6c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:43.966776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84bba6-a815-457a-b0c4-49bb081a6c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:43.969008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84bba6-a815-457a-b0c4-49bb081a6c56 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:43.980560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1298/2596] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:27:44.283239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5313d9-f593-4790-b4aa-bd25c60108ba None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:44.284899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-fa5313d9-f593-4790-b4aa-bd25c60108ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:44.303866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5313d9-f593-4790-b4aa-bd25c60108ba None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:44.309582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:44.310505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:44.310800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:44.311077 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] PATH_INFO: `/v3/projects/02c6777eea5b4606a88ec18223b6a6d0` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:44.312332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c6777eea5b4606a88ec18223b6a6d0)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:44.314416 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:44.326896 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:44.367897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c6777eea5b4606a88ec18223b6a6d0', 'request_id': 'req-c8ec9927-30b9-42aa-9930-65fe91257952'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.377199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c6777eea5b4606a88ec18223b6a6d0', 'request_id': 'req-c8ec9927-30b9-42aa-9930-65fe91257952'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.383859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c6777eea5b4606a88ec18223b6a6d0', 'request_id': 'req-c8ec9927-30b9-42aa-9930-65fe91257952'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.395627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] The token cache is being invalidate because project 02c6777eea5b4606a88ec18223b6a6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:44.395917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.396660 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.398374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1299/2597] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:44 2022] DELETE /identity/v3/projects/02c6777eea5b4606a88ec18223b6a6d0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:44.402504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84bba6-a815-457a-b0c4-49bb081a6c56 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:44.413217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84bba6-a815-457a-b0c4-49bb081a6c56 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:44.477188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:44.478240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:44.478787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:44.479082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] PATH_INFO: `/v3/users/ca9c046b41584527b2dffa1ddc5d2a70` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:44.480515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca9c046b41584527b2dffa1ddc5d2a70)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:44.483687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:44.535356 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca9c046b41584527b2dffa1ddc5d2a70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.544337 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca9c046b41584527b2dffa1ddc5d2a70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.557993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca9c046b41584527b2dffa1ddc5d2a70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.558771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca9c046b41584527b2dffa1ddc5d2a70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.560264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-90387483-3426-4e14-a0f6-3c24c96a7dec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca9c046b41584527b2dffa1ddc5d2a70'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:44.568207 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:44.574191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1299/2598] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:44 2022] DELETE /identity/v3/users/ca9c046b41584527b2dffa1ddc5d2a70 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:44.865075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:44.866668 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:44.884071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec9927-30b9-42aa-9930-65fe91257952 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:44.949041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:44.950705 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:44.950997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:44.951278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] PATH_INFO: `/v3/projects/255b3a59ca174b668c62b4c8a3400c74` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:44.952512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=255b3a59ca174b668c62b4c8a3400c74)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:44.954495 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.008197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '255b3a59ca174b668c62b4c8a3400c74', 'request_id': 'req-c69907e6-b73b-4c7e-8e38-be2635da4e18'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:45.017437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '255b3a59ca174b668c62b4c8a3400c74', 'request_id': 'req-c69907e6-b73b-4c7e-8e38-be2635da4e18'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:45.024464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '255b3a59ca174b668c62b4c8a3400c74', 'request_id': 'req-c69907e6-b73b-4c7e-8e38-be2635da4e18'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:45.036025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] The token cache is being invalidate because project 255b3a59ca174b668c62b4c8a3400c74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:45.036285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:45.036993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c69907e6-b73b-4c7e-8e38-be2635da4e18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:45.038743 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1300/2599] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:44 2022] DELETE /identity/v3/projects/255b3a59ca174b668c62b4c8a3400c74 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:45.043899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ff499-3710-4c22-a2ad-9f9db5b54a26 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.044377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ff499-3710-4c22-a2ad-9f9db5b54a26 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.044720 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ff499-3710-4c22-a2ad-9f9db5b54a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.066010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-2f2ff499-3710-4c22-a2ad-9f9db5b54a26 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:45.067940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2f2ff499-3710-4c22-a2ad-9f9db5b54a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:45.139879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1300/2600] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:45.148357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dc2bb-7c0a-4ee0-b385-f8c2d17ad324 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.148716 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dc2bb-7c0a-4ee0-b385-f8c2d17ad324 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.149011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dc2bb-7c0a-4ee0-b385-f8c2d17ad324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.169786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-240dc2bb-7c0a-4ee0-b385-f8c2d17ad324 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:45.171046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-240dc2bb-7c0a-4ee0-b385-f8c2d17ad324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:45.179669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1301/2601] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:45.184521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ff499-3710-4c22-a2ad-9f9db5b54a26 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.195181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ff499-3710-4c22-a2ad-9f9db5b54a26 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.201105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.201797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.202086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.202452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.203104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:45.203415 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.205094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.218346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1301/2602] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:27:45.223113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dc2bb-7c0a-4ee0-b385-f8c2d17ad324 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.234102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dc2bb-7c0a-4ee0-b385-f8c2d17ad324 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.239372 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0002-eb67-412c-907f-75e732e1b0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.240049 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0002-eb67-412c-907f-75e732e1b0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.240330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0002-eb67-412c-907f-75e732e1b0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.240605 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d0002-eb67-412c-907f-75e732e1b0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.241300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d0002-eb67-412c-907f-75e732e1b0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.243264 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d0002-eb67-412c-907f-75e732e1b0a1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.276691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1302/2603] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:45 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:45.281463 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.292115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c6d55-710b-4592-94f9-9a7c5fa3dd89 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.352946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573a0aa-2d96-4418-9a1d-50e826a4484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.353844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a0aa-2d96-4418-9a1d-50e826a4484a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.354128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a0aa-2d96-4418-9a1d-50e826a4484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.354388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a0aa-2d96-4418-9a1d-50e826a4484a admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.355252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573a0aa-2d96-4418-9a1d-50e826a4484a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.357869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573a0aa-2d96-4418-9a1d-50e826a4484a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.383572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1302/2604] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:45 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:45.384862 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0002-eb67-412c-907f-75e732e1b0a1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.391974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:45.399304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d0002-eb67-412c-907f-75e732e1b0a1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.406023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb9630-ab48-4984-a18c-630c9c86422b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.406757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb9630-ab48-4984-a18c-630c9c86422b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.407041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb9630-ab48-4984-a18c-630c9c86422b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.407315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb9630-ab48-4984-a18c-630c9c86422b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.408076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb9630-ab48-4984-a18c-630c9c86422b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.409712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb9630-ab48-4984-a18c-630c9c86422b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.420905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1303/2605] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:45.425508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573a0aa-2d96-4418-9a1d-50e826a4484a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.437151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573a0aa-2d96-4418-9a1d-50e826a4484a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.443010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c84573-f5e8-4fc4-9d45-de6070a05284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.443687 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84573-f5e8-4fc4-9d45-de6070a05284 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.443968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84573-f5e8-4fc4-9d45-de6070a05284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.444246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c84573-f5e8-4fc4-9d45-de6070a05284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.444979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c84573-f5e8-4fc4-9d45-de6070a05284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.446644 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c84573-f5e8-4fc4-9d45-de6070a05284 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.458016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1303/2606] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:45.462503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb9630-ab48-4984-a18c-630c9c86422b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.473835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb9630-ab48-4984-a18c-630c9c86422b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.479591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5cf333-0447-4b62-987d-3ade3edcab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.480458 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cf333-0447-4b62-987d-3ade3edcab97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.480776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cf333-0447-4b62-987d-3ade3edcab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.481202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cf333-0447-4b62-987d-3ade3edcab97 admin admin] PATH_INFO: `/v3/projects/1b061001972b47ac8761c10653d2469b/users/b6e3f4c9c2424907b34222221f154ce6/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.497712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5cf333-0447-4b62-987d-3ade3edcab97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b061001972b47ac8761c10653d2469b, user_id=b6e3f4c9c2424907b34222221f154ce6, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.500363 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5cf333-0447-4b62-987d-3ade3edcab97 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.515814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1304/2607] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:45 2022] PUT /identity/v3/projects/1b061001972b47ac8761c10653d2469b/users/b6e3f4c9c2424907b34222221f154ce6/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:45.526715 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d8a09-dc88-470a-9a71-d7d6fc7e9693 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.527012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d8a09-dc88-470a-9a71-d7d6fc7e9693 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.527272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d8a09-dc88-470a-9a71-d7d6fc7e9693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.560809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-0c7d8a09-dc88-470a-9a71-d7d6fc7e9693 None None] MFA Rules not processed for user `b6e3f4c9c2424907b34222221f154ce6`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:45.561964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0c7d8a09-dc88-470a-9a71-d7d6fc7e9693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:45.621311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:45.635482 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1304/2608] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:45.640442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5cf333-0447-4b62-987d-3ade3edcab97 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.704588 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5cf333-0447-4b62-987d-3ade3edcab97 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.710178 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d0f1c-8fe9-457e-9a77-5d4a1cb98c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.710861 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d0f1c-8fe9-457e-9a77-5d4a1cb98c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.711138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d0f1c-8fe9-457e-9a77-5d4a1cb98c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.711408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d0f1c-8fe9-457e-9a77-5d4a1cb98c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.712144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d0f1c-8fe9-457e-9a77-5d4a1cb98c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.714099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d0f1c-8fe9-457e-9a77-5d4a1cb98c97 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.741966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1305/2609] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:45 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:45.746080 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d8a09-dc88-470a-9a71-d7d6fc7e9693 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.757423 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d8a09-dc88-470a-9a71-d7d6fc7e9693 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.814573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f1fb2-ace3-4eb1-978e-916877bfc952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.815271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f1fb2-ace3-4eb1-978e-916877bfc952 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.815553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f1fb2-ace3-4eb1-978e-916877bfc952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.815840 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f1fb2-ace3-4eb1-978e-916877bfc952 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.816522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f1fb2-ace3-4eb1-978e-916877bfc952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.818511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f1fb2-ace3-4eb1-978e-916877bfc952 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.840879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1305/2610] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:45 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:45.846683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d0f1c-8fe9-457e-9a77-5d4a1cb98c97 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.859012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d0f1c-8fe9-457e-9a77-5d4a1cb98c97 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.864980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb09196-3e14-4beb-ae15-730b1211ff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.865640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb09196-3e14-4beb-ae15-730b1211ff4a admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.865955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb09196-3e14-4beb-ae15-730b1211ff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.866231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb09196-3e14-4beb-ae15-730b1211ff4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.866969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb09196-3e14-4beb-ae15-730b1211ff4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.868598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb09196-3e14-4beb-ae15-730b1211ff4a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.880159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1306/2611] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:45.886336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f1fb2-ace3-4eb1-978e-916877bfc952 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.898869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f1fb2-ace3-4eb1-978e-916877bfc952 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:45.906535 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d45c01-0597-49a2-8288-58bf94c07f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:45.907736 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d45c01-0597-49a2-8288-58bf94c07f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:45.908066 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d45c01-0597-49a2-8288-58bf94c07f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:45.908342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d45c01-0597-49a2-8288-58bf94c07f8d admin admin] PATH_INFO: `/v3/projects/87aeb385f5614a66b049fb75dd3fb830/users/7971b20979184692a07ebdf2a7f5c9f5/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:45.926887 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d45c01-0597-49a2-8288-58bf94c07f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=87aeb385f5614a66b049fb75dd3fb830, user_id=7971b20979184692a07ebdf2a7f5c9f5, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:45.929594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d45c01-0597-49a2-8288-58bf94c07f8d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:45.947853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1306/2612] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:45 2022] PUT /identity/v3/projects/87aeb385f5614a66b049fb75dd3fb830/users/7971b20979184692a07ebdf2a7f5c9f5/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:45.952571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb09196-3e14-4beb-ae15-730b1211ff4a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:45.962871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb09196-3e14-4beb-ae15-730b1211ff4a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:46.023667 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07160e-754d-40c4-ae31-3ac73deb522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:46.024351 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07160e-754d-40c4-ae31-3ac73deb522d admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:46.024632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07160e-754d-40c4-ae31-3ac73deb522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:46.024918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07160e-754d-40c4-ae31-3ac73deb522d admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:46.025627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07160e-754d-40c4-ae31-3ac73deb522d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:46.027297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07160e-754d-40c4-ae31-3ac73deb522d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:46.038497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1307/2613] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:46.043031 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d45c01-0597-49a2-8288-58bf94c07f8d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:46.054473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d45c01-0597-49a2-8288-58bf94c07f8d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:46.060934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:46.061869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:46.062154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:46.062432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] PATH_INFO: `/v3/projects/87aeb385f5614a66b049fb75dd3fb830/users/7971b20979184692a07ebdf2a7f5c9f5/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:46.064658 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87aeb385f5614a66b049fb75dd3fb830, user_id=7971b20979184692a07ebdf2a7f5c9f5, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:46.067386 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:46.079338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1307/2614] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:46 2022] PUT /identity/v3/projects/87aeb385f5614a66b049fb75dd3fb830/users/7971b20979184692a07ebdf2a7f5c9f5/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:46.091646 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadd50d-90c3-49aa-9572-31add16c444b None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:46.092336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadd50d-90c3-49aa-9572-31add16c444b None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:46.092336 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dadd50d-90c3-49aa-9572-31add16c444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:46.122057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9dadd50d-90c3-49aa-9572-31add16c444b None None] MFA Rules not processed for user `7971b20979184692a07ebdf2a7f5c9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:46.123311 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9dadd50d-90c3-49aa-9572-31add16c444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:46.150737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:46.197726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1308/2615] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:46.208784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:46.210570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:46.274325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091730d-1447-40d4-b7cc-eb03e956bd14 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:46.278358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:46.279010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:46.279297 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:46.279574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:46.289923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:46.291458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:46.302278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1308/2616] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:46.349649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadd50d-90c3-49aa-9572-31add16c444b None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:46.351213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9dadd50d-90c3-49aa-9572-31add16c444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:46.365487 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dadd50d-90c3-49aa-9572-31add16c444b None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:46.369453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c859d3-eee5-413a-8bb9-045bac4621a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:46.370141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c859d3-eee5-413a-8bb9-045bac4621a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:46.370417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c859d3-eee5-413a-8bb9-045bac4621a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:46.370693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c859d3-eee5-413a-8bb9-045bac4621a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:46.435459 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c859d3-eee5-413a-8bb9-045bac4621a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:46.436938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c859d3-eee5-413a-8bb9-045bac4621a2 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:46.449834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1309/2617] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:48.743343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:48.745401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:48.815810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:48.838721 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda81c3-6c67-44c2-8fe0-d5619fbd5d63 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:48.846113 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:48.847019 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:48.847293 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:48.847559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] PATH_INFO: `/v3/users/ba84a806bd5a499d8c5dc5c81a91fb0e` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:48.850572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba84a806bd5a499d8c5dc5c81a91fb0e)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:48.853155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:48.910484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba84a806bd5a499d8c5dc5c81a91fb0e'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:48.920208 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba84a806bd5a499d8c5dc5c81a91fb0e'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:48.934650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba84a806bd5a499d8c5dc5c81a91fb0e'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:48.935308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba84a806bd5a499d8c5dc5c81a91fb0e'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:48.935879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba84a806bd5a499d8c5dc5c81a91fb0e'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:48.948800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1309/2618] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:48 2022] DELETE /identity/v3/users/ba84a806bd5a499d8c5dc5c81a91fb0e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:48.953597 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6c6c-442a-47a5-89d7-49ba35e046cf None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:48.954177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6c6c-442a-47a5-89d7-49ba35e046cf None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:48.954177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca6c6c-442a-47a5-89d7-49ba35e046cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:48.977762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-f3ca6c6c-442a-47a5-89d7-49ba35e046cf None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:48.979691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f3ca6c6c-442a-47a5-89d7-49ba35e046cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:49.050735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1310/2619] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:49.063015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:49.064676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:49.126724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec47d57-7f00-4294-b98c-59561c3d2cae admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:49.130817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc228e61-7ccd-4f0f-9891-7ebe3860ee2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:49.131390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc228e61-7ccd-4f0f-9891-7ebe3860ee2f service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:49.131689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc228e61-7ccd-4f0f-9891-7ebe3860ee2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:49.131964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc228e61-7ccd-4f0f-9891-7ebe3860ee2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:49.142651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc228e61-7ccd-4f0f-9891-7ebe3860ee2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:49.144111 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc228e61-7ccd-4f0f-9891-7ebe3860ee2f service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:49.154209 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1310/2620] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:49.530713 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca6c6c-442a-47a5-89d7-49ba35e046cf None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:49.532169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f3ca6c6c-442a-47a5-89d7-49ba35e046cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:49.547033 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca6c6c-442a-47a5-89d7-49ba35e046cf None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:49.552966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:49.554765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:49.555148 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:49.555535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] PATH_INFO: `/v3/projects/3ded73fff49a4611acb1ef2937601fa4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:49.557051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ded73fff49a4611acb1ef2937601fa4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:49.560190 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:49.611024 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ded73fff49a4611acb1ef2937601fa4', 'request_id': 'req-c71975a8-29e3-4439-a9ed-5d2fa73445a7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.619739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ded73fff49a4611acb1ef2937601fa4', 'request_id': 'req-c71975a8-29e3-4439-a9ed-5d2fa73445a7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.620963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:49.628480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ded73fff49a4611acb1ef2937601fa4', 'request_id': 'req-c71975a8-29e3-4439-a9ed-5d2fa73445a7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.641216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] The token cache is being invalidate because project 3ded73fff49a4611acb1ef2937601fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:49.641422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.642202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.643873 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1311/2621] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:49 2022] DELETE /identity/v3/projects/3ded73fff49a4611acb1ef2937601fa4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:49.648236 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc228e61-7ccd-4f0f-9891-7ebe3860ee2f service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:49.658832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc228e61-7ccd-4f0f-9891-7ebe3860ee2f service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:49.723188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:49.724531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:49.724531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:49.724732 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] PATH_INFO: `/v3/users/3e10d2cd060a43bcb56e5d0421203644` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:49.725942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e10d2cd060a43bcb56e5d0421203644)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:49.727948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:49.742103 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:49.799324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e10d2cd060a43bcb56e5d0421203644'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.808587 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e10d2cd060a43bcb56e5d0421203644'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.825060 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e10d2cd060a43bcb56e5d0421203644'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.825874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e10d2cd060a43bcb56e5d0421203644'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.826474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5825240b-bba6-477e-9b95-7f5cfac6c0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e10d2cd060a43bcb56e5d0421203644'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:49.840646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1311/2622] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:49 2022] DELETE /identity/v3/users/3e10d2cd060a43bcb56e5d0421203644 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:49.908213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:49.909990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:49.923864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71975a8-29e3-4439-a9ed-5d2fa73445a7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:49.993419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:49.994378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:49.994665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:49.994943 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] PATH_INFO: `/v3/users/b6e3f4c9c2424907b34222221f154ce6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:49.996466 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e3f4c9c2424907b34222221f154ce6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:49.998376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:50.069334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e3f4c9c2424907b34222221f154ce6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.079427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e3f4c9c2424907b34222221f154ce6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.092655 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e3f4c9c2424907b34222221f154ce6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.093460 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e3f4c9c2424907b34222221f154ce6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.094219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e3f4c9c2424907b34222221f154ce6'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.109473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1312/2623] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:49 2022] DELETE /identity/v3/users/b6e3f4c9c2424907b34222221f154ce6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:50.114459 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5219-1375-4496-b63f-7daba98844af None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:50.115191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5219-1375-4496-b63f-7daba98844af None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:50.115588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5219-1375-4496-b63f-7daba98844af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:50.135796 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-5bff5219-1375-4496-b63f-7daba98844af None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:50.137189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5bff5219-1375-4496-b63f-7daba98844af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:50.146528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:50.148122 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:50.207635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1312/2624] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:50.218173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff5219-1375-4496-b63f-7daba98844af None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:50.219804 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5bff5219-1375-4496-b63f-7daba98844af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:50.221512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10f776-fad7-46db-833c-4adadc96c8f0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:50.226223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:50.227104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:50.227392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:50.227670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] PATH_INFO: `/v3/projects/c4a755a68e9f4e98955e05170c854d89` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:50.228870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a755a68e9f4e98955e05170c854d89)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:50.230946 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:50.238769 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:50.288072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff5219-1375-4496-b63f-7daba98844af None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:50.291869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a755a68e9f4e98955e05170c854d89', 'request_id': 'req-8e2a8268-c8f0-488d-9779-15100abb93e4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.301471 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a755a68e9f4e98955e05170c854d89', 'request_id': 'req-8e2a8268-c8f0-488d-9779-15100abb93e4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.308574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a755a68e9f4e98955e05170c854d89', 'request_id': 'req-8e2a8268-c8f0-488d-9779-15100abb93e4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.321118 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] The token cache is being invalidate because project c4a755a68e9f4e98955e05170c854d89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:50.321408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.322191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.324197 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1313/2625] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:50 2022] DELETE /identity/v3/projects/c4a755a68e9f4e98955e05170c854d89 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:50.328506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:50.329940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:50.341024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:50.348547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:50.349344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:50.349730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:50.350072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:50.352785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:50.407404 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a8268-c8f0-488d-9779-15100abb93e4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:50.412216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:50.413123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:50.413696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:50.413823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] PATH_INFO: `/v3/users/5ec5063b921f407582197671a3cce6a8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:50.415083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ec5063b921f407582197671a3cce6a8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:50.417070 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:50.438520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:50.441451 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:50.455547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1313/2626] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:50.482875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ec5063b921f407582197671a3cce6a8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.492857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ec5063b921f407582197671a3cce6a8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.518524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ec5063b921f407582197671a3cce6a8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.519391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ec5063b921f407582197671a3cce6a8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.520368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ec5063b921f407582197671a3cce6a8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.537239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1314/2627] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:50 2022] DELETE /identity/v3/users/5ec5063b921f407582197671a3cce6a8 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:50.831016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:50.832495 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:50.846467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1552a68-02e3-48c3-be48-3cd5e27c267a service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:50.913505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:50.914442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:50.914756 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:50.915020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] PATH_INFO: `/v3/projects/1b061001972b47ac8761c10653d2469b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:50.916281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b061001972b47ac8761c10653d2469b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:50.919090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:50.934540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:50.936538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:50.953831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e5fa91-59a9-4e9e-9473-9e498e23b571 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:50.964426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:50.966359 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:50.966656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:50.966938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] PATH_INFO: `/v3/projects/ef50e15f9cf14b8bb72ea9520aa0dfa3` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:50.968354 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef50e15f9cf14b8bb72ea9520aa0dfa3)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:50.970514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:50.975951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b061001972b47ac8761c10653d2469b', 'request_id': 'req-3863e714-6272-4429-812c-e27d3aff74e8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.979206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:50.982889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b061001972b47ac8761c10653d2469b', 'request_id': 'req-3863e714-6272-4429-812c-e27d3aff74e8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:50.993186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b061001972b47ac8761c10653d2469b', 'request_id': 'req-3863e714-6272-4429-812c-e27d3aff74e8'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.004867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] The token cache is being invalidate because project 1b061001972b47ac8761c10653d2469b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:51.005229 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.005922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.007491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1314/2628] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:50 2022] DELETE /identity/v3/projects/1b061001972b47ac8761c10653d2469b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:51.011401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.013333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.028627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef50e15f9cf14b8bb72ea9520aa0dfa3', 'request_id': 'req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.036813 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef50e15f9cf14b8bb72ea9520aa0dfa3', 'request_id': 'req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.044036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef50e15f9cf14b8bb72ea9520aa0dfa3', 'request_id': 'req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.045247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:51.062100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] The token cache is being invalidate because project ef50e15f9cf14b8bb72ea9520aa0dfa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:51.062445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.063036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-aeffb1af-0f9b-44fb-a225-7e6e97fb8a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.068922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1315/2629] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:50 2022] DELETE /identity/v3/projects/ef50e15f9cf14b8bb72ea9520aa0dfa3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:51.070802 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da05c-58bd-4e7f-abe1-c877e7837761 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.071089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da05c-58bd-4e7f-abe1-c877e7837761 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.071870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da05c-58bd-4e7f-abe1-c877e7837761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.091338 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b62da05c-58bd-4e7f-abe1-c877e7837761 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:51.092570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b62da05c-58bd-4e7f-abe1-c877e7837761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.099192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863e714-6272-4429-812c-e27d3aff74e8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.173973 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1316/2630] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:51.182951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d1ecc-3502-468a-a1d3-25cdf3cd47b7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.184279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d1ecc-3502-468a-a1d3-25cdf3cd47b7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.184530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.184788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d1ecc-3502-468a-a1d3-25cdf3cd47b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.186041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.186041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.186258 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] PATH_INFO: `/v3/users/7971b20979184692a07ebdf2a7f5c9f5` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.187358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7971b20979184692a07ebdf2a7f5c9f5)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.189358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.218935 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0b3d1ecc-3502-468a-a1d3-25cdf3cd47b7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:51.220479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0b3d1ecc-3502-468a-a1d3-25cdf3cd47b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.230158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1317/2631] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:51.235133 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d1ecc-3502-468a-a1d3-25cdf3cd47b7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.244638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d1ecc-3502-468a-a1d3-25cdf3cd47b7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.244898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7971b20979184692a07ebdf2a7f5c9f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.251888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.252766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.253053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.253332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.254126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7971b20979184692a07ebdf2a7f5c9f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.254294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:51.254526 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.256192 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.268046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7971b20979184692a07ebdf2a7f5c9f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.268805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1318/2632] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:27:51.268945 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7971b20979184692a07ebdf2a7f5c9f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.270448 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7971b20979184692a07ebdf2a7f5c9f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.273430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.274948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.284794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1315/2633] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:51 2022] DELETE /identity/v3/users/7971b20979184692a07ebdf2a7f5c9f5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:51.289765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2225-daa7-4a1d-bb46-f67ae9b19198 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.347866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7172bc-5de3-49ba-9ee1-d619becdb021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.348676 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7172bc-5de3-49ba-9ee1-d619becdb021 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.348834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7172bc-5de3-49ba-9ee1-d619becdb021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.349078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7172bc-5de3-49ba-9ee1-d619becdb021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.349830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7172bc-5de3-49ba-9ee1-d619becdb021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.351786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7172bc-5de3-49ba-9ee1-d619becdb021 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.379880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1319/2634] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:51.384106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.395879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc9222-7e64-427d-a1c7-e2208d20970c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.449164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:51.461903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea31507-c1a5-46b0-befb-8a278c124f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.462595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea31507-c1a5-46b0-befb-8a278c124f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.462879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea31507-c1a5-46b0-befb-8a278c124f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.463155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea31507-c1a5-46b0-befb-8a278c124f81 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.463866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea31507-c1a5-46b0-befb-8a278c124f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.466449 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea31507-c1a5-46b0-befb-8a278c124f81 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.489562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1316/2635] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:51.493956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7172bc-5de3-49ba-9ee1-d619becdb021 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.500167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:51.506486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7172bc-5de3-49ba-9ee1-d619becdb021 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.513430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a637d35-5ea0-4abe-b89e-3831b26c77a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.514358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a637d35-5ea0-4abe-b89e-3831b26c77a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.514933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a637d35-5ea0-4abe-b89e-3831b26c77a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.515289 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a637d35-5ea0-4abe-b89e-3831b26c77a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.516167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a637d35-5ea0-4abe-b89e-3831b26c77a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.517817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a637d35-5ea0-4abe-b89e-3831b26c77a9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.529341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1320/2636] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:51.533494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea31507-c1a5-46b0-befb-8a278c124f81 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.545716 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea31507-c1a5-46b0-befb-8a278c124f81 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.551479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.552224 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.552724 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.553006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.553853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.555662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.568782 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1317/2637] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:51.573161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a637d35-5ea0-4abe-b89e-3831b26c77a9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.585322 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a637d35-5ea0-4abe-b89e-3831b26c77a9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.592104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.593002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe9575-2e9c-4605-a0b1-70c66da44517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.594151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.594294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe9575-2e9c-4605-a0b1-70c66da44517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.594504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe9575-2e9c-4605-a0b1-70c66da44517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.594765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe9575-2e9c-4605-a0b1-70c66da44517 admin admin] PATH_INFO: `/v3/projects/6e81930c6a9e4cd98f48cfe672c509af/users/c50bba5970f54de49a433fe158b895fa/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.614263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe9575-2e9c-4605-a0b1-70c66da44517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e81930c6a9e4cd98f48cfe672c509af, user_id=c50bba5970f54de49a433fe158b895fa, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.615662 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa54f04-f041-4abc-917d-f6c7aca71c94 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.617480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe9575-2e9c-4605-a0b1-70c66da44517 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.621861 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.623303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.624190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.624479 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] PATH_INFO: `/v3/projects/87aeb385f5614a66b049fb75dd3fb830` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.625820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] RBAC: Authorizing `identity:delete_project(project_id=87aeb385f5614a66b049fb75dd3fb830)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.627808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.657817 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1321/2638] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:51 2022] PUT /identity/v3/projects/6e81930c6a9e4cd98f48cfe672c509af/users/c50bba5970f54de49a433fe158b895fa/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:51.660381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf51d37-5555-4b91-b192-77327677735f None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.662069 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf51d37-5555-4b91-b192-77327677735f None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.662532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf51d37-5555-4b91-b192-77327677735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.695455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-2cf51d37-5555-4b91-b192-77327677735f None None] MFA Rules not processed for user `c50bba5970f54de49a433fe158b895fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:51.696600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2cf51d37-5555-4b91-b192-77327677735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.702532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87aeb385f5614a66b049fb75dd3fb830', 'request_id': 'req-1b852598-2109-4c53-9df2-f256754ba5be'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.711323 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87aeb385f5614a66b049fb75dd3fb830', 'request_id': 'req-1b852598-2109-4c53-9df2-f256754ba5be'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.721212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87aeb385f5614a66b049fb75dd3fb830', 'request_id': 'req-1b852598-2109-4c53-9df2-f256754ba5be'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.733862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] The token cache is being invalidate because project 87aeb385f5614a66b049fb75dd3fb830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:51.734151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.734833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-1b852598-2109-4c53-9df2-f256754ba5be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:51.736433 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1318/2639] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:51 2022] DELETE /identity/v3/projects/87aeb385f5614a66b049fb75dd3fb830 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:51.741647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac02f4-5544-4df7-aa43-c635b3cebc9c None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.741968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac02f4-5544-4df7-aa43-c635b3cebc9c None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.742247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac02f4-5544-4df7-aa43-c635b3cebc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.763611 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9fac02f4-5544-4df7-aa43-c635b3cebc9c None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:51.767612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9fac02f4-5544-4df7-aa43-c635b3cebc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.768937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1322/2640] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:51.780632 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf51d37-5555-4b91-b192-77327677735f None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.782063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-2cf51d37-5555-4b91-b192-77327677735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.843908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1319/2641] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:51.845752 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad877e-fa34-439b-a5be-55620c4b3424 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.845939 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad877e-fa34-439b-a5be-55620c4b3424 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.846076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf51d37-5555-4b91-b192-77327677735f None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.846219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad877e-fa34-439b-a5be-55620c4b3424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.850019 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f0eaa-3ae4-4000-8c54-7e2f90a7bb35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd661615b2dd94d9db6183f580ab0fee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.850727 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0eaa-3ae4-4000-8c54-7e2f90a7bb35 service glance] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.851001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0eaa-3ae4-4000-8c54-7e2f90a7bb35 service glance] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.851284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0eaa-3ae4-4000-8c54-7e2f90a7bb35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.860976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f0eaa-3ae4-4000-8c54-7e2f90a7bb35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.862453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31f0eaa-3ae4-4000-8c54-7e2f90a7bb35 service glance] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.867132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-35ad877e-fa34-439b-a5be-55620c4b3424 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:51.868362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-35ad877e-fa34-439b-a5be-55620c4b3424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:51.872920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1323/2642] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:27:51.881348 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1320/2643] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:51.882740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f0eaa-3ae4-4000-8c54-7e2f90a7bb35 service glance] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.894548 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f0eaa-3ae4-4000-8c54-7e2f90a7bb35 service glance] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.899454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.900125 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.900406 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.900807 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.901542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:27:51.901894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.903752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.916949 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1324/2644] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:27:51.921719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad877e-fa34-439b-a5be-55620c4b3424 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.932117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad877e-fa34-439b-a5be-55620c4b3424 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:51.938601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a912a-74bc-42d2-b810-666ab6c334bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:51.939281 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a912a-74bc-42d2-b810-666ab6c334bb admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:51.939562 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a912a-74bc-42d2-b810-666ab6c334bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:51.939865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a912a-74bc-42d2-b810-666ab6c334bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:51.940593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a912a-74bc-42d2-b810-666ab6c334bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:51.942661 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a912a-74bc-42d2-b810-666ab6c334bb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:51.971873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1321/2645] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:51.976143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:51.987231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033e5f6-4ba5-4102-af13-2e446c420397 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.038199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:52.052879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce51616-4b0b-43a3-9a45-461d0897c51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.053557 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce51616-4b0b-43a3-9a45-461d0897c51a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.053874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce51616-4b0b-43a3-9a45-461d0897c51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.054156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce51616-4b0b-43a3-9a45-461d0897c51a admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.054838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce51616-4b0b-43a3-9a45-461d0897c51a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.064723 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce51616-4b0b-43a3-9a45-461d0897c51a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.088281 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1325/2646] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:51 2022] POST /identity/v3/users => generated 305 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:27:52.094328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a912a-74bc-42d2-b810-666ab6c334bb admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.107377 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a912a-74bc-42d2-b810-666ab6c334bb admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.112445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af4662-dac5-4315-92cc-9fb8f1f1d2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.113122 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af4662-dac5-4315-92cc-9fb8f1f1d2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.113404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af4662-dac5-4315-92cc-9fb8f1f1d2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.113709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af4662-dac5-4315-92cc-9fb8f1f1d2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.114452 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af4662-dac5-4315-92cc-9fb8f1f1d2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.116079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af4662-dac5-4315-92cc-9fb8f1f1d2c7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.128204 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1322/2647] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:52.132524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce51616-4b0b-43a3-9a45-461d0897c51a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.143837 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce51616-4b0b-43a3-9a45-461d0897c51a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.152644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd7eda-c691-4d5d-819d-8d4d649ecb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.153618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7eda-c691-4d5d-819d-8d4d649ecb51 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.153937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7eda-c691-4d5d-819d-8d4d649ecb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.154225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7eda-c691-4d5d-819d-8d4d649ecb51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.155145 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd7eda-c691-4d5d-819d-8d4d649ecb51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.157232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd7eda-c691-4d5d-819d-8d4d649ecb51 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.175380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1326/2648] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:27:52.177396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af4662-dac5-4315-92cc-9fb8f1f1d2c7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.188949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af4662-dac5-4315-92cc-9fb8f1f1d2c7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.193959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7cd101-1162-4ea9-b6f0-9b79ce59572c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.195155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cd101-1162-4ea9-b6f0-9b79ce59572c admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.195494 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cd101-1162-4ea9-b6f0-9b79ce59572c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.195981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cd101-1162-4ea9-b6f0-9b79ce59572c admin admin] PATH_INFO: `/v3/projects/999b5cb25948443aa48579131c0a3e90/users/40a2dbb82b67411d9bf4b894fc8bb4a4/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.211269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:52.214954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7cd101-1162-4ea9-b6f0-9b79ce59572c admin admin] RBAC: Authorizing `identity:create_grant(project_id=999b5cb25948443aa48579131c0a3e90, user_id=40a2dbb82b67411d9bf4b894fc8bb4a4, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.218875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7cd101-1162-4ea9-b6f0-9b79ce59572c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.234705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1323/2649] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:52 2022] PUT /identity/v3/projects/999b5cb25948443aa48579131c0a3e90/users/40a2dbb82b67411d9bf4b894fc8bb4a4/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:27:52.247321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f32c1-34cd-4f05-b647-dcf31116bfa1 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.247740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f32c1-34cd-4f05-b647-dcf31116bfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.248242 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f32c1-34cd-4f05-b647-dcf31116bfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.289798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-b08f32c1-34cd-4f05-b647-dcf31116bfa1 None None] MFA Rules not processed for user `40a2dbb82b67411d9bf4b894fc8bb4a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:52.291799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-b08f32c1-34cd-4f05-b647-dcf31116bfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:52.400023 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1327/2650] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:52.402093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7cd101-1162-4ea9-b6f0-9b79ce59572c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.497418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7cd101-1162-4ea9-b6f0-9b79ce59572c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.505726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634f9af-35a0-4556-babb-5c437c366762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.506575 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f9af-35a0-4556-babb-5c437c366762 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.506982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f9af-35a0-4556-babb-5c437c366762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.507499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f9af-35a0-4556-babb-5c437c366762 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.508830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634f9af-35a0-4556-babb-5c437c366762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.512667 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634f9af-35a0-4556-babb-5c437c366762 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.549037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1324/2651] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:27:52 2022] POST /identity/v3/projects => generated 409 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:52.550611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f32c1-34cd-4f05-b647-dcf31116bfa1 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.562960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f32c1-34cd-4f05-b647-dcf31116bfa1 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.641602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab757e6c-17f0-4c30-84d5-74b97c3e79f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.642325 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab757e6c-17f0-4c30-84d5-74b97c3e79f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.642637 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab757e6c-17f0-4c30-84d5-74b97c3e79f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.642919 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab757e6c-17f0-4c30-84d5-74b97c3e79f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.643675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab757e6c-17f0-4c30-84d5-74b97c3e79f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.645926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab757e6c-17f0-4c30-84d5-74b97c3e79f2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.669754 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1328/2652] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:27:52 2022] POST /identity/v3/users => generated 311 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:27:52.673307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634f9af-35a0-4556-babb-5c437c366762 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.684176 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634f9af-35a0-4556-babb-5c437c366762 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.690792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d89cd-db57-4bbb-b39a-adc418b5500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.692226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d89cd-db57-4bbb-b39a-adc418b5500a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.692510 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d89cd-db57-4bbb-b39a-adc418b5500a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.692792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d89cd-db57-4bbb-b39a-adc418b5500a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.693561 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d89cd-db57-4bbb-b39a-adc418b5500a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.695217 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d89cd-db57-4bbb-b39a-adc418b5500a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.708685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1325/2653] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:52.713156 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab757e6c-17f0-4c30-84d5-74b97c3e79f2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.723948 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab757e6c-17f0-4c30-84d5-74b97c3e79f2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.729539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984a60d-4bf5-46a8-b53b-12cee8e1955e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.730545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984a60d-4bf5-46a8-b53b-12cee8e1955e admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.731549 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984a60d-4bf5-46a8-b53b-12cee8e1955e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.731860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984a60d-4bf5-46a8-b53b-12cee8e1955e admin admin] PATH_INFO: `/v3/projects/b2155f54f93642c38f65a425dc0bfb67/users/d640f91dcc4d4954afac745d08c59cdb/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.751573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984a60d-4bf5-46a8-b53b-12cee8e1955e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2155f54f93642c38f65a425dc0bfb67, user_id=d640f91dcc4d4954afac745d08c59cdb, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.754543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984a60d-4bf5-46a8-b53b-12cee8e1955e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.769438 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1329/2654] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:52 2022] PUT /identity/v3/projects/b2155f54f93642c38f65a425dc0bfb67/users/d640f91dcc4d4954afac745d08c59cdb/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:52.773607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d89cd-db57-4bbb-b39a-adc418b5500a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.784652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d89cd-db57-4bbb-b39a-adc418b5500a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.863897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428860e-696b-481b-9041-37d76711f201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.864951 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428860e-696b-481b-9041-37d76711f201 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.866177 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428860e-696b-481b-9041-37d76711f201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.866469 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428860e-696b-481b-9041-37d76711f201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.867719 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428860e-696b-481b-9041-37d76711f201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.870092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428860e-696b-481b-9041-37d76711f201 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.886922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1326/2655] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:27:52.891583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984a60d-4bf5-46a8-b53b-12cee8e1955e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:52.902410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984a60d-4bf5-46a8-b53b-12cee8e1955e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:52.907651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:52.908692 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:52.908812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:52.909172 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] PATH_INFO: `/v3/projects/b2155f54f93642c38f65a425dc0bfb67/users/d640f91dcc4d4954afac745d08c59cdb/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:52.911422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2155f54f93642c38f65a425dc0bfb67, user_id=d640f91dcc4d4954afac745d08c59cdb, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:52.914569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:52.931078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1330/2656] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:27:52 2022] PUT /identity/v3/projects/b2155f54f93642c38f65a425dc0bfb67/users/d640f91dcc4d4954afac745d08c59cdb/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:53.003964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08429fb7-46d5-4ca1-948c-c6a06ea15906 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:53.004322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08429fb7-46d5-4ca1-948c-c6a06ea15906 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:53.004434 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08429fb7-46d5-4ca1-948c-c6a06ea15906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:53.035979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-08429fb7-46d5-4ca1-948c-c6a06ea15906 None None] MFA Rules not processed for user `d640f91dcc4d4954afac745d08c59cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:53.037640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-08429fb7-46d5-4ca1-948c-c6a06ea15906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:53.090092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:53.119560 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1327/2657] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:53 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:27:53.131856 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:53.133735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:53.144394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:53.195425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c787af-100b-4f80-b44c-f5ce78310184 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:53.199205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:53.199953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:53.200241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:53.200519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:53.210377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:53.211850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:53.230866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1331/2658] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:53 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:27:53.407428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08429fb7-46d5-4ca1-948c-c6a06ea15906 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:53.409075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-08429fb7-46d5-4ca1-948c-c6a06ea15906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:53.423409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08429fb7-46d5-4ca1-948c-c6a06ea15906 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:53.427966 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760c129-90ab-4dcf-92bd-dab2462a0378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:53.428627 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c129-90ab-4dcf-92bd-dab2462a0378 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:53.428912 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c129-90ab-4dcf-92bd-dab2462a0378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:53.429187 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760c129-90ab-4dcf-92bd-dab2462a0378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:53.492302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760c129-90ab-4dcf-92bd-dab2462a0378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:53.493816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760c129-90ab-4dcf-92bd-dab2462a0378 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:53.504673 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1328/2659] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:53 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:27:58.904986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:58.906656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:58.981819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd6c10-19aa-447b-b947-6f6676eda9e5 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:58.986989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:58.987886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:58.988173 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:58.988444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] PATH_INFO: `/v3/users/e6d2c4a6ec604c2c88ec349a3802e7a0` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:58.989905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6d2c4a6ec604c2c88ec349a3802e7a0)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:58.991830 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:59.052091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6d2c4a6ec604c2c88ec349a3802e7a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.062879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6d2c4a6ec604c2c88ec349a3802e7a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.078284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6d2c4a6ec604c2c88ec349a3802e7a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.079182 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6d2c4a6ec604c2c88ec349a3802e7a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.079839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6d2c4a6ec604c2c88ec349a3802e7a0'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.092581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1332/2660] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:58 2022] DELETE /identity/v3/users/e6d2c4a6ec604c2c88ec349a3802e7a0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:59.098551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880e4c-d87a-4e03-8ed8-25516aab0af4 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:59.099138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880e4c-d87a-4e03-8ed8-25516aab0af4 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:59.099138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880e4c-d87a-4e03-8ed8-25516aab0af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:59.118359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-de880e4c-d87a-4e03-8ed8-25516aab0af4 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:27:59.119622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-de880e4c-d87a-4e03-8ed8-25516aab0af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:59.188964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1329/2661] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:27:59.198517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:59.200180 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:59.237082 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:27:59.270330 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717863b5-2ed2-4c7f-9132-93f7ec5cdcba admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:59.274512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6dfbd-94d4-4662-be1f-17ebf7d6825a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:59.275177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6dfbd-94d4-4662-be1f-17ebf7d6825a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:59.275463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6dfbd-94d4-4662-be1f-17ebf7d6825a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:59.275763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6dfbd-94d4-4662-be1f-17ebf7d6825a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:59.286280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6dfbd-94d4-4662-be1f-17ebf7d6825a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:59.287790 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6dfbd-94d4-4662-be1f-17ebf7d6825a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:59.298782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1333/2662] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:27:59.629714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de880e4c-d87a-4e03-8ed8-25516aab0af4 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:59.634502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-de880e4c-d87a-4e03-8ed8-25516aab0af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:27:59.646630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de880e4c-d87a-4e03-8ed8-25516aab0af4 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:59.653885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:59.654859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:59.655201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:59.655546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] PATH_INFO: `/v3/projects/136509c568a64771b7617842464aab9f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:59.664098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=136509c568a64771b7617842464aab9f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:59.667351 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:59.722025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '136509c568a64771b7617842464aab9f', 'request_id': 'req-5826d021-7a2f-4c10-8db9-71ebcea520c5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.728970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '136509c568a64771b7617842464aab9f', 'request_id': 'req-5826d021-7a2f-4c10-8db9-71ebcea520c5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.738517 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '136509c568a64771b7617842464aab9f', 'request_id': 'req-5826d021-7a2f-4c10-8db9-71ebcea520c5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.750496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] The token cache is being invalidate because project 136509c568a64771b7617842464aab9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:27:59.750793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.751571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.753322 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1330/2663] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:27:59 2022] DELETE /identity/v3/projects/136509c568a64771b7617842464aab9f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:27:59.757900 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6dfbd-94d4-4662-be1f-17ebf7d6825a service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:27:59.776006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6dfbd-94d4-4662-be1f-17ebf7d6825a service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:27:59.836586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:27:59.837558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:27:59.837940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:27:59.838221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] PATH_INFO: `/v3/users/ed7589d70ed64394b2e5ab12783feca8` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:27:59.839525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7589d70ed64394b2e5ab12783feca8)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:27:59.841865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:27:59.902367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7589d70ed64394b2e5ab12783feca8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.913531 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7589d70ed64394b2e5ab12783feca8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.933841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7589d70ed64394b2e5ab12783feca8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.934780 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7589d70ed64394b2e5ab12783feca8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.935493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-69f0b210-93d1-4f79-bdc3-19faaa7fdad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7589d70ed64394b2e5ab12783feca8'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:27:59.952671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1334/2664] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:27:59 2022] DELETE /identity/v3/users/ed7589d70ed64394b2e5ab12783feca8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:00.265769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:00.267272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:00.283090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826d021-7a2f-4c10-8db9-71ebcea520c5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:00.291093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:00.357024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:00.358319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.358625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.358986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] PATH_INFO: `/v3/projects/49f022fa9fd4474280ddde89aff11c22` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.360241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f022fa9fd4474280ddde89aff11c22)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:00.362559 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:00.416179 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f022fa9fd4474280ddde89aff11c22', 'request_id': 'req-d348d86b-a577-4da4-aa44-b72945ea07db'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:00.424870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f022fa9fd4474280ddde89aff11c22', 'request_id': 'req-d348d86b-a577-4da4-aa44-b72945ea07db'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:00.434885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f022fa9fd4474280ddde89aff11c22', 'request_id': 'req-d348d86b-a577-4da4-aa44-b72945ea07db'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:00.454691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] The token cache is being invalidate because project 49f022fa9fd4474280ddde89aff11c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:00.455024 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:00.455751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d348d86b-a577-4da4-aa44-b72945ea07db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:00.457487 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1331/2665] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:00 2022] DELETE /identity/v3/projects/49f022fa9fd4474280ddde89aff11c22 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:00.462984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6b97-c9cf-40ae-9b8b-224cd5199ff7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.463376 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6b97-c9cf-40ae-9b8b-224cd5199ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.463664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f6b97-c9cf-40ae-9b8b-224cd5199ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.485555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-6c6f6b97-c9cf-40ae-9b8b-224cd5199ff7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:00.487029 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6c6f6b97-c9cf-40ae-9b8b-224cd5199ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:00.539389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:00.556430 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1335/2666] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:00.564654 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22584586-fe95-4256-90f6-d0d93dc465bb None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.564964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22584586-fe95-4256-90f6-d0d93dc465bb None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.565221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22584586-fe95-4256-90f6-d0d93dc465bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.585821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-22584586-fe95-4256-90f6-d0d93dc465bb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:00.587216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-22584586-fe95-4256-90f6-d0d93dc465bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:00.595776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1332/2667] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:00.600798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6b97-c9cf-40ae-9b8b-224cd5199ff7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:00.611881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f6b97-c9cf-40ae-9b8b-224cd5199ff7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:00.617018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:00.617736 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.618043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.618304 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.619043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:00.619346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:00.621108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:00.642265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1336/2668] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:28:00.646327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22584586-fe95-4256-90f6-d0d93dc465bb None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:00.656280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22584586-fe95-4256-90f6-d0d93dc465bb None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:00.661922 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf05991-237d-4643-85a4-e50f4f0990d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:00.662793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf05991-237d-4643-85a4-e50f4f0990d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.663320 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf05991-237d-4643-85a4-e50f4f0990d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.663778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf05991-237d-4643-85a4-e50f4f0990d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.664624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf05991-237d-4643-85a4-e50f4f0990d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:00.667555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf05991-237d-4643-85a4-e50f4f0990d7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:00.706381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1333/2669] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:00 2022] POST /identity/v3/projects => generated 369 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:00.707735 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:00.726219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9f5bf-42fe-4323-96d7-4dc62a44334e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:00.790089 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ebc3a-e719-4732-9aef-57c2fa2a372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:00.790835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ebc3a-e719-4732-9aef-57c2fa2a372a admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.791211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ebc3a-e719-4732-9aef-57c2fa2a372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.791737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ebc3a-e719-4732-9aef-57c2fa2a372a admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.792957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ebc3a-e719-4732-9aef-57c2fa2a372a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:00.794805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ebc3a-e719-4732-9aef-57c2fa2a372a admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:00.824562 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1337/2670] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:00 2022] POST /identity/v3/users => generated 285 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:00.828699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf05991-237d-4643-85a4-e50f4f0990d7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:00.839012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf05991-237d-4643-85a4-e50f4f0990d7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:00.843725 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa249b0-16e8-4e3d-9a34-d63577b6d65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:00.844388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa249b0-16e8-4e3d-9a34-d63577b6d65b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.844675 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa249b0-16e8-4e3d-9a34-d63577b6d65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.844954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa249b0-16e8-4e3d-9a34-d63577b6d65b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.845704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa249b0-16e8-4e3d-9a34-d63577b6d65b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:00.847336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa249b0-16e8-4e3d-9a34-d63577b6d65b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:00.859013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1334/2671] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:00.863436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ebc3a-e719-4732-9aef-57c2fa2a372a admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:00.874428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ebc3a-e719-4732-9aef-57c2fa2a372a admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:00.879142 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45a6ca-0c01-4fa9-b9bd-c20f9fa76279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:00.879815 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a6ca-0c01-4fa9-b9bd-c20f9fa76279 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.880100 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a6ca-0c01-4fa9-b9bd-c20f9fa76279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.880378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45a6ca-0c01-4fa9-b9bd-c20f9fa76279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.881108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45a6ca-0c01-4fa9-b9bd-c20f9fa76279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:00.882758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45a6ca-0c01-4fa9-b9bd-c20f9fa76279 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:00.893129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1338/2672] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:00.897590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa249b0-16e8-4e3d-9a34-d63577b6d65b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:00.908483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa249b0-16e8-4e3d-9a34-d63577b6d65b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:00.913490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e873a8-b472-48f0-a24f-0827806965e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:00.914409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e873a8-b472-48f0-a24f-0827806965e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.914729 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e873a8-b472-48f0-a24f-0827806965e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.915040 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e873a8-b472-48f0-a24f-0827806965e5 admin admin] PATH_INFO: `/v3/projects/393f709ad5854a8fb43adbc6bb3ce691/users/842cb52a0d52423c969e8a4f9f5d51fd/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.930739 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e873a8-b472-48f0-a24f-0827806965e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=393f709ad5854a8fb43adbc6bb3ce691, user_id=842cb52a0d52423c969e8a4f9f5d51fd, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:00.933780 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e873a8-b472-48f0-a24f-0827806965e5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:00.949872 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1335/2673] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:00 2022] PUT /identity/v3/projects/393f709ad5854a8fb43adbc6bb3ce691/users/842cb52a0d52423c969e8a4f9f5d51fd/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:00.960922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7273e-d31b-4c0b-b038-efac5b08bbef None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:00.961478 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7273e-d31b-4c0b-b038-efac5b08bbef None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:00.961579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7273e-d31b-4c0b-b038-efac5b08bbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:00.994093 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0bf7273e-d31b-4c0b-b038-efac5b08bbef None None] MFA Rules not processed for user `842cb52a0d52423c969e8a4f9f5d51fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:00.995349 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0bf7273e-d31b-4c0b-b038-efac5b08bbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:01.063606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1339/2674] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:01.069465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e873a8-b472-48f0-a24f-0827806965e5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:01.114749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:01.141554 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e873a8-b472-48f0-a24f-0827806965e5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:01.146712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab17cad-c731-4c64-b197-b104370e85b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:01.148126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab17cad-c731-4c64-b197-b104370e85b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.148126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab17cad-c731-4c64-b197-b104370e85b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.148126 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab17cad-c731-4c64-b197-b104370e85b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.148621 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab17cad-c731-4c64-b197-b104370e85b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:01.150640 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab17cad-c731-4c64-b197-b104370e85b3 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:01.182324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1336/2675] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:01 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:01.186907 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7273e-d31b-4c0b-b038-efac5b08bbef None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:01.198609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7273e-d31b-4c0b-b038-efac5b08bbef None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:01.245046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:01.262977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703da62-2dc4-4ff5-98cd-984f53177390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:01.263706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703da62-2dc4-4ff5-98cd-984f53177390 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.264015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703da62-2dc4-4ff5-98cd-984f53177390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.264310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703da62-2dc4-4ff5-98cd-984f53177390 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.265077 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703da62-2dc4-4ff5-98cd-984f53177390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:01.267261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703da62-2dc4-4ff5-98cd-984f53177390 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:01.290482 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1340/2676] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:01 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:01.295507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab17cad-c731-4c64-b197-b104370e85b3 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:01.306961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab17cad-c731-4c64-b197-b104370e85b3 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:01.312672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832e98d-92ed-4ae1-82ae-b780428b8420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:01.313343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832e98d-92ed-4ae1-82ae-b780428b8420 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.313622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832e98d-92ed-4ae1-82ae-b780428b8420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.313946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832e98d-92ed-4ae1-82ae-b780428b8420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.314689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832e98d-92ed-4ae1-82ae-b780428b8420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:01.316315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832e98d-92ed-4ae1-82ae-b780428b8420 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:01.328123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1337/2677] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:01.332437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703da62-2dc4-4ff5-98cd-984f53177390 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:01.343473 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703da62-2dc4-4ff5-98cd-984f53177390 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:01.348934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c1158-dd5c-459a-9111-7a3943bd6d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:01.349854 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c1158-dd5c-459a-9111-7a3943bd6d15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.350139 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c1158-dd5c-459a-9111-7a3943bd6d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.350412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c1158-dd5c-459a-9111-7a3943bd6d15 admin admin] PATH_INFO: `/v3/projects/91158e583fe44913a81f8086b1137adf/users/ab46ddd6d89f43c9945e3b4f36623100/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.374384 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c1158-dd5c-459a-9111-7a3943bd6d15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91158e583fe44913a81f8086b1137adf, user_id=ab46ddd6d89f43c9945e3b4f36623100, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:01.377129 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c1158-dd5c-459a-9111-7a3943bd6d15 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:01.395439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1341/2678] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:01 2022] PUT /identity/v3/projects/91158e583fe44913a81f8086b1137adf/users/ab46ddd6d89f43c9945e3b4f36623100/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:01.402242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832e98d-92ed-4ae1-82ae-b780428b8420 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:01.413350 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832e98d-92ed-4ae1-82ae-b780428b8420 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:01.490106 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8088dff-d6fc-4f50-81ff-31b9cd946b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:01.490801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8088dff-d6fc-4f50-81ff-31b9cd946b4e admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.491087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8088dff-d6fc-4f50-81ff-31b9cd946b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.491366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8088dff-d6fc-4f50-81ff-31b9cd946b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.492101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8088dff-d6fc-4f50-81ff-31b9cd946b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:01.493791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8088dff-d6fc-4f50-81ff-31b9cd946b4e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:01.504534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1338/2679] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:01.508791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c1158-dd5c-459a-9111-7a3943bd6d15 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:01.520407 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c1158-dd5c-459a-9111-7a3943bd6d15 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:01.525255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:01.526205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.526484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.526759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] PATH_INFO: `/v3/projects/91158e583fe44913a81f8086b1137adf/users/ab46ddd6d89f43c9945e3b4f36623100/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.529130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91158e583fe44913a81f8086b1137adf, user_id=ab46ddd6d89f43c9945e3b4f36623100, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:01.531841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:01.542506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1342/2680] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:01 2022] PUT /identity/v3/projects/91158e583fe44913a81f8086b1137adf/users/ab46ddd6d89f43c9945e3b4f36623100/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:01.553013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec982e-f402-4fc5-9983-12883daff5fc None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.553308 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec982e-f402-4fc5-9983-12883daff5fc None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.553582 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec982e-f402-4fc5-9983-12883daff5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.588242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-87ec982e-f402-4fc5-9983-12883daff5fc None None] MFA Rules not processed for user `ab46ddd6d89f43c9945e3b4f36623100`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:01.589730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87ec982e-f402-4fc5-9983-12883daff5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:01.670894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1339/2681] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:01.682523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:01.684554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:01.747586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9fb31-a671-4c37-b775-ecf7f1e77e50 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:01.751906 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:01.752567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.752847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.753119 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.763414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:01.764920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:01.774955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1343/2682] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:01.822837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec982e-f402-4fc5-9983-12883daff5fc None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:01.825251 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87ec982e-f402-4fc5-9983-12883daff5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:01.840701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec982e-f402-4fc5-9983-12883daff5fc None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:01.846571 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb08f0-827d-494d-b815-15dfbf913513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:01.847626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb08f0-827d-494d-b815-15dfbf913513 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:01.848115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb08f0-827d-494d-b815-15dfbf913513 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:01.848467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb08f0-827d-494d-b815-15dfbf913513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:01.889396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:01.924202 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb08f0-827d-494d-b815-15dfbf913513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:01.926039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb08f0-827d-494d-b815-15dfbf913513 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:01.936935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1340/2683] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:02.106371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:02.107882 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:02.198926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6510aa-526d-4f2b-b1e1-f4cef09f3c7a service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:02.204348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:02.205358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:02.205534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:02.205810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] PATH_INFO: `/v3/users/40a2dbb82b67411d9bf4b894fc8bb4a4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:02.207366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40a2dbb82b67411d9bf4b894fc8bb4a4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:02.209183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:02.278734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:02.287442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40a2dbb82b67411d9bf4b894fc8bb4a4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:02.299408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40a2dbb82b67411d9bf4b894fc8bb4a4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:02.315591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40a2dbb82b67411d9bf4b894fc8bb4a4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:02.316645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40a2dbb82b67411d9bf4b894fc8bb4a4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:02.318310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40a2dbb82b67411d9bf4b894fc8bb4a4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:02.340002 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1344/2684] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:02 2022] DELETE /identity/v3/users/40a2dbb82b67411d9bf4b894fc8bb4a4 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:02.344995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa7a9f-ea64-4303-ba57-5b20c54481aa None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:02.345593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa7a9f-ea64-4303-ba57-5b20c54481aa None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:02.346037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa7a9f-ea64-4303-ba57-5b20c54481aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:02.368905 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-21fa7a9f-ea64-4303-ba57-5b20c54481aa None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:02.370396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-21fa7a9f-ea64-4303-ba57-5b20c54481aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:02.442077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1341/2685] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:02.451916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:02.453625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:02.541094 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e2752-1e5d-4310-bc30-9dada6ffeb53 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:02.551016 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5e019-a602-472c-ba8e-97094516587a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:02.551721 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5e019-a602-472c-ba8e-97094516587a service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:02.551993 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5e019-a602-472c-ba8e-97094516587a service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:02.552280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5e019-a602-472c-ba8e-97094516587a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:02.564492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5e019-a602-472c-ba8e-97094516587a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:02.566527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5e019-a602-472c-ba8e-97094516587a service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:02.585428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1345/2686] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:02.916245 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa7a9f-ea64-4303-ba57-5b20c54481aa None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:02.918771 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-21fa7a9f-ea64-4303-ba57-5b20c54481aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:02.936986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa7a9f-ea64-4303-ba57-5b20c54481aa None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:02.943029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:02.944062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:02.944332 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:02.944615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] PATH_INFO: `/v3/projects/999b5cb25948443aa48579131c0a3e90` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:02.946325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999b5cb25948443aa48579131c0a3e90)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:02.948257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:03.010313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999b5cb25948443aa48579131c0a3e90', 'request_id': 'req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.017701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999b5cb25948443aa48579131c0a3e90', 'request_id': 'req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.026860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999b5cb25948443aa48579131c0a3e90', 'request_id': 'req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.039636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] The token cache is being invalidate because project 999b5cb25948443aa48579131c0a3e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:03.039779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.040438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.042707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1342/2687] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:02 2022] DELETE /identity/v3/projects/999b5cb25948443aa48579131c0a3e90 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:03.046573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5e019-a602-472c-ba8e-97094516587a service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:03.056970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5e019-a602-472c-ba8e-97094516587a service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:03.123777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:03.124773 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:03.125120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:03.125463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] PATH_INFO: `/v3/users/d640f91dcc4d4954afac745d08c59cdb` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:03.127008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d640f91dcc4d4954afac745d08c59cdb)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:03.129041 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:03.186552 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd640f91dcc4d4954afac745d08c59cdb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.196240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd640f91dcc4d4954afac745d08c59cdb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.204542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:03.209912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd640f91dcc4d4954afac745d08c59cdb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.211025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd640f91dcc4d4954afac745d08c59cdb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.211595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a2a9dcb1-fee3-4cac-9be6-80e0817b4297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd640f91dcc4d4954afac745d08c59cdb'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.225194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1346/2688] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:03 2022] DELETE /identity/v3/users/d640f91dcc4d4954afac745d08c59cdb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:03.531120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:03.533058 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:03.554441 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a068bfcb-88b8-42e2-88d1-d12a6431d2f5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:03.633536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:03.644749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:03.645873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:03.646240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:03.646975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] PATH_INFO: `/v3/projects/b2155f54f93642c38f65a425dc0bfb67` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:03.649703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2155f54f93642c38f65a425dc0bfb67)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:03.653705 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:03.708583 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2155f54f93642c38f65a425dc0bfb67', 'request_id': 'req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.715841 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2155f54f93642c38f65a425dc0bfb67', 'request_id': 'req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.724215 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2155f54f93642c38f65a425dc0bfb67', 'request_id': 'req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.736701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] The token cache is being invalidate because project b2155f54f93642c38f65a425dc0bfb67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:03.736998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.737801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2e2e05c6-4dbb-4e45-ab1c-145c8acee28b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:03.739630 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1343/2689] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:03 2022] DELETE /identity/v3/projects/b2155f54f93642c38f65a425dc0bfb67 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:03.746464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bb477-e0d0-4495-bd9b-33b6030bec40 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:03.746757 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bb477-e0d0-4495-bd9b-33b6030bec40 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:03.747045 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bb477-e0d0-4495-bd9b-33b6030bec40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:03.775340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-4f7bb477-e0d0-4495-bd9b-33b6030bec40 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:03.776881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-4f7bb477-e0d0-4495-bd9b-33b6030bec40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:03.858481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1347/2690] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:03.866873 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe30c5-3a16-4ce2-a73c-5c731f1c7067 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:03.867163 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe30c5-3a16-4ce2-a73c-5c731f1c7067 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:03.868247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe30c5-3a16-4ce2-a73c-5c731f1c7067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:03.886618 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-87fe30c5-3a16-4ce2-a73c-5c731f1c7067 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:03.892607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-87fe30c5-3a16-4ce2-a73c-5c731f1c7067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:03.901864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1344/2691] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:03.906586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bb477-e0d0-4495-bd9b-33b6030bec40 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:03.916763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bb477-e0d0-4495-bd9b-33b6030bec40 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:03.921456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:03.922160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:03.922443 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:03.922717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:03.923440 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:03.923747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:03.925842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:03.949701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1348/2692] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:28:03.954546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe30c5-3a16-4ce2-a73c-5c731f1c7067 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:03.964917 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe30c5-3a16-4ce2-a73c-5c731f1c7067 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:03.970292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd352cb-4b3f-4d66-928b-8851affb9765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:03.971888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd352cb-4b3f-4d66-928b-8851affb9765 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:03.972168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd352cb-4b3f-4d66-928b-8851affb9765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:03.972439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd352cb-4b3f-4d66-928b-8851affb9765 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:03.973194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd352cb-4b3f-4d66-928b-8851affb9765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:03.975911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd352cb-4b3f-4d66-928b-8851affb9765 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:04.016616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1345/2693] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:03 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:04.017841 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:04.028540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89714b38-6997-4766-82fc-70e8a515f46c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:04.115115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07d45-12cb-4de0-9fbe-e6e4090774a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:04.115996 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07d45-12cb-4de0-9fbe-e6e4090774a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:04.116383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07d45-12cb-4de0-9fbe-e6e4090774a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:04.116806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07d45-12cb-4de0-9fbe-e6e4090774a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:04.118348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a07d45-12cb-4de0-9fbe-e6e4090774a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:04.122868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a07d45-12cb-4de0-9fbe-e6e4090774a5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:04.146324 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1349/2694] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:04 2022] POST /identity/v3/users => generated 282 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:04.150695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd352cb-4b3f-4d66-928b-8851affb9765 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:04.164770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd352cb-4b3f-4d66-928b-8851affb9765 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:04.170880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63f7da-7c76-4b22-8d2c-1387b4702e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:04.171712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f7da-7c76-4b22-8d2c-1387b4702e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:04.171931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f7da-7c76-4b22-8d2c-1387b4702e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:04.172260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63f7da-7c76-4b22-8d2c-1387b4702e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:04.172946 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63f7da-7c76-4b22-8d2c-1387b4702e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:04.174615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63f7da-7c76-4b22-8d2c-1387b4702e1a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:04.189762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1346/2695] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:04.191164 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07d45-12cb-4de0-9fbe-e6e4090774a5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:04.202752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07d45-12cb-4de0-9fbe-e6e4090774a5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:04.209765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feebcd0-7e33-4389-a428-97fe14dac1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:04.210519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feebcd0-7e33-4389-a428-97fe14dac1cc admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:04.210719 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feebcd0-7e33-4389-a428-97fe14dac1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:04.211081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feebcd0-7e33-4389-a428-97fe14dac1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:04.212329 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feebcd0-7e33-4389-a428-97fe14dac1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:04.213728 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feebcd0-7e33-4389-a428-97fe14dac1cc admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:04.217601 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc096a7-b85b-41b9-b22b-557eabbf70a5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:04.218239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc096a7-b85b-41b9-b22b-557eabbf70a5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:04.218731 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc096a7-b85b-41b9-b22b-557eabbf70a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:04.226057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1350/2696] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:04.231004 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feebcd0-7e33-4389-a428-97fe14dac1cc admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:04.243361 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feebcd0-7e33-4389-a428-97fe14dac1cc admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:04.243730 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7cc096a7-b85b-41b9-b22b-557eabbf70a5 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:04.244970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7cc096a7-b85b-41b9-b22b-557eabbf70a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:04.248590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc7158-2f32-4d1b-a042-69b8cd9e5481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:04.250734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc7158-2f32-4d1b-a042-69b8cd9e5481 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:04.250734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc7158-2f32-4d1b-a042-69b8cd9e5481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:04.250734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc7158-2f32-4d1b-a042-69b8cd9e5481 admin admin] PATH_INFO: `/v3/projects/b7d42244ad82492aadf1f51d7ad74f33/users/288bcc4d12374626b1b9802a811a61f4/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:04.266539 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc7158-2f32-4d1b-a042-69b8cd9e5481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d42244ad82492aadf1f51d7ad74f33, user_id=288bcc4d12374626b1b9802a811a61f4, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:04.269469 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc7158-2f32-4d1b-a042-69b8cd9e5481 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:04.282976 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1351/2697] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:04 2022] PUT /identity/v3/projects/b7d42244ad82492aadf1f51d7ad74f33/users/288bcc4d12374626b1b9802a811a61f4/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:04.295768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcdc91-a4eb-4154-9351-6ae57bd9fedc None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:04.296177 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcdc91-a4eb-4154-9351-6ae57bd9fedc None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:04.296497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bcdc91-a4eb-4154-9351-6ae57bd9fedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:04.329514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1347/2698] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:04.338985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-53bcdc91-a4eb-4154-9351-6ae57bd9fedc None None] MFA Rules not processed for user `288bcc4d12374626b1b9802a811a61f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:04.340927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53bcdc91-a4eb-4154-9351-6ae57bd9fedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:04.346190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc096a7-b85b-41b9-b22b-557eabbf70a5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:04.347955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7cc096a7-b85b-41b9-b22b-557eabbf70a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:04.361673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:04.425329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:04.439426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1352/2699] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:04.451303 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcdc91-a4eb-4154-9351-6ae57bd9fedc None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:04.452701 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53bcdc91-a4eb-4154-9351-6ae57bd9fedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:04.454264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc096a7-b85b-41b9-b22b-557eabbf70a5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:04.519131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bcdc91-a4eb-4154-9351-6ae57bd9fedc None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:04.523025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da054dd-d1a1-464b-85b2-47ceb605a4a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:04.523690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da054dd-d1a1-464b-85b2-47ceb605a4a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:04.524009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da054dd-d1a1-464b-85b2-47ceb605a4a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:04.524283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da054dd-d1a1-464b-85b2-47ceb605a4a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:04.529078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982afd5-56c2-4798-b570-d8c076e923c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:04.529798 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982afd5-56c2-4798-b570-d8c076e923c8 service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:04.530081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982afd5-56c2-4798-b570-d8c076e923c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:04.530359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982afd5-56c2-4798-b570-d8c076e923c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:04.534505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da054dd-d1a1-464b-85b2-47ceb605a4a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:04.535958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da054dd-d1a1-464b-85b2-47ceb605a4a2 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:04.549886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1353/2700] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:04 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:28:04.595035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982afd5-56c2-4798-b570-d8c076e923c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:04.597409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982afd5-56c2-4798-b570-d8c076e923c8 service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:04.607814 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1348/2701] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:28:05.254290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61b5f2-c37e-46ef-9675-e92a00c86845 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:05.255123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61b5f2-c37e-46ef-9675-e92a00c86845 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:05.255559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61b5f2-c37e-46ef-9675-e92a00c86845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:05.275356 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-8a61b5f2-c37e-46ef-9675-e92a00c86845 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:05.276507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8a61b5f2-c37e-46ef-9675-e92a00c86845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:05.284141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1354/2702] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 18 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:05.296978 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982afd5-56c2-4798-b570-d8c076e923c8 service swift] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:05.307766 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982afd5-56c2-4798-b570-d8c076e923c8 service swift] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:05.313214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a13592d-401c-4642-a3e8-c82dc6df9e5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:05.313867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13592d-401c-4642-a3e8-c82dc6df9e5e service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:05.314149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13592d-401c-4642-a3e8-c82dc6df9e5e service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:05.314420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a13592d-401c-4642-a3e8-c82dc6df9e5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:05.323164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13592d-401c-4642-a3e8-c82dc6df9e5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:05.324867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a13592d-401c-4642-a3e8-c82dc6df9e5e service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:05.333476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1349/2703] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:28:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 18 14:28:05.480547 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61b5f2-c37e-46ef-9675-e92a00c86845 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:05.482494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8a61b5f2-c37e-46ef-9675-e92a00c86845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:05.553190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed0e16-7d03-45ba-a15f-8e6376e441b0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:05.553483 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed0e16-7d03-45ba-a15f-8e6376e441b0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:05.553806 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed0e16-7d03-45ba-a15f-8e6376e441b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:05.560091 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61b5f2-c37e-46ef-9675-e92a00c86845 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:05.566259 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:05.567606 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:05.567975 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:05.568288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] PATH_INFO: `/v3/users/842cb52a0d52423c969e8a4f9f5d51fd` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:05.569945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=842cb52a0d52423c969e8a4f9f5d51fd)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:05.571766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:05.573474 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6eed0e16-7d03-45ba-a15f-8e6376e441b0 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:05.575746 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6eed0e16-7d03-45ba-a15f-8e6376e441b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:05.586718 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1350/2704] 10.222.0.71 () {58 vars in 1009 bytes} [Mon Jul 18 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:05.613270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed0e16-7d03-45ba-a15f-8e6376e441b0 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:05.625588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed0e16-7d03-45ba-a15f-8e6376e441b0 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:05.626105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842cb52a0d52423c969e8a4f9f5d51fd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:05.634810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:05.635136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376a2c3-78b2-4ee0-876e-bad3a1f2bc1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:05.635136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376a2c3-78b2-4ee0-876e-bad3a1f2bc1b service swift] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:05.635136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376a2c3-78b2-4ee0-876e-bad3a1f2bc1b service swift] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:05.635136 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376a2c3-78b2-4ee0-876e-bad3a1f2bc1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:05.635511 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842cb52a0d52423c969e8a4f9f5d51fd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:05.645049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376a2c3-78b2-4ee0-876e-bad3a1f2bc1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:05.647171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376a2c3-78b2-4ee0-876e-bad3a1f2bc1b service swift] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:05.649917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842cb52a0d52423c969e8a4f9f5d51fd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:05.650670 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842cb52a0d52423c969e8a4f9f5d51fd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:05.651283 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842cb52a0d52423c969e8a4f9f5d51fd'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:05.656397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1351/2705] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:28:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 14:28:05.666310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1355/2706] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:05 2022] DELETE /identity/v3/users/842cb52a0d52423c969e8a4f9f5d51fd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:05.671695 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28e7bd-41e8-4f5f-a947-c53e1e80eaa0 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:05.672083 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28e7bd-41e8-4f5f-a947-c53e1e80eaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:05.672408 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e28e7bd-41e8-4f5f-a947-c53e1e80eaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:05.691144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3e28e7bd-41e8-4f5f-a947-c53e1e80eaa0 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:05.692364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3e28e7bd-41e8-4f5f-a947-c53e1e80eaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:05.752581 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=100 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:05.762854 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1352/2707] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:05.773015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:05.774567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:05.843381 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcf086-a3c9-4904-9f98-643a7ef2c034 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:05.852544 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:05.853246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:05.853725 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:05.853984 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:05.865362 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:05.866977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:05.878015 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1356/2708] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:06.017610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fb29-a692-4e92-a6b5-e8c1440d30d2 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:06.017938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fb29-a692-4e92-a6b5-e8c1440d30d2 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:06.018220 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fb29-a692-4e92-a6b5-e8c1440d30d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:06.037848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-3eb3fb29-a692-4e92-a6b5-e8c1440d30d2 None None] MFA Rules not processed for user `1ecd34d877d4476d86e0e8b4ae9c56c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:06.039092 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3eb3fb29-a692-4e92-a6b5-e8c1440d30d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:06.118622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1353/2709] 10.222.0.71 () {58 vars in 967 bytes} [Mon Jul 18 14:28:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:06.133227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:06.134895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:06.188894 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3fb29-a692-4e92-a6b5-e8c1440d30d2 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:06.190492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-3eb3fb29-a692-4e92-a6b5-e8c1440d30d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:06.222689 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3fb29-a692-4e92-a6b5-e8c1440d30d2 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:06.228340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:06.229343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:06.229768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:06.230585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] PATH_INFO: `/v3/projects/393f709ad5854a8fb43adbc6bb3ce691` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:06.232037 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=393f709ad5854a8fb43adbc6bb3ce691)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:06.234532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:06.270412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94eb9f-f541-4475-8109-8d46ed217d87 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:06.279193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b465f-1e93-496f-9cee-3561f181204a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '203e654f95bf45e683b0ccc4ba687bfc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:06.279990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b465f-1e93-496f-9cee-3561f181204a service swift] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:06.280367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b465f-1e93-496f-9cee-3561f181204a service swift] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:06.280917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39b465f-1e93-496f-9cee-3561f181204a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:06.295380 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b465f-1e93-496f-9cee-3561f181204a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:06.297398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39b465f-1e93-496f-9cee-3561f181204a service swift] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:06.300194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393f709ad5854a8fb43adbc6bb3ce691', 'request_id': 'req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.306923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1357/2710] 10.222.0.71 () {60 vars in 1384 bytes} [Mon Jul 18 14:28:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 18 14:28:06.307610 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393f709ad5854a8fb43adbc6bb3ce691', 'request_id': 'req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.318953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393f709ad5854a8fb43adbc6bb3ce691', 'request_id': 'req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.337093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] The token cache is being invalidate because project 393f709ad5854a8fb43adbc6bb3ce691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:06.337304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.338656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.341197 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1354/2711] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:06 2022] DELETE /identity/v3/projects/393f709ad5854a8fb43adbc6bb3ce691 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:06.345235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b465f-1e93-496f-9cee-3561f181204a service swift] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:06.357587 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39b465f-1e93-496f-9cee-3561f181204a service swift] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:06.431165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:06.432432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:06.432786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:06.433146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] PATH_INFO: `/v3/users/ab46ddd6d89f43c9945e3b4f36623100` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:06.434784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab46ddd6d89f43c9945e3b4f36623100)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:06.436997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:06.440833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:06.514957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab46ddd6d89f43c9945e3b4f36623100'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.525290 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab46ddd6d89f43c9945e3b4f36623100'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.542570 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab46ddd6d89f43c9945e3b4f36623100'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.543990 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab46ddd6d89f43c9945e3b4f36623100'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.548869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-9d2a7758-7a52-436b-ba7c-9be3f183fa07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab46ddd6d89f43c9945e3b4f36623100'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:06.566250 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1358/2712] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:06 2022] DELETE /identity/v3/users/ab46ddd6d89f43c9945e3b4f36623100 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:06.876421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:06.878285 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:06.893594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455df1d-c7a9-4f35-b6a3-2609df6aeaaa admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:06.955604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:06.956488 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:06.956769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:06.957051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] PATH_INFO: `/v3/projects/91158e583fe44913a81f8086b1137adf` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:06.958290 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91158e583fe44913a81f8086b1137adf)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:06.961394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.004635 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:07.022919 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91158e583fe44913a81f8086b1137adf', 'request_id': 'req-d45071b7-46f0-4610-9a22-30d2d93b3121'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:07.029379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91158e583fe44913a81f8086b1137adf', 'request_id': 'req-d45071b7-46f0-4610-9a22-30d2d93b3121'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:07.037347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91158e583fe44913a81f8086b1137adf', 'request_id': 'req-d45071b7-46f0-4610-9a22-30d2d93b3121'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:07.048586 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] The token cache is being invalidate because project 91158e583fe44913a81f8086b1137adf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:07.048881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:07.049528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-d45071b7-46f0-4610-9a22-30d2d93b3121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:07.051186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1355/2713] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:06 2022] DELETE /identity/v3/projects/91158e583fe44913a81f8086b1137adf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:07.056741 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7f8e5-c486-4a42-abfd-a2fd5d7e8fd9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.057134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7f8e5-c486-4a42-abfd-a2fd5d7e8fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.057507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7f8e5-c486-4a42-abfd-a2fd5d7e8fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.076352 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-61b7f8e5-c486-4a42-abfd-a2fd5d7e8fd9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:07.077584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-61b7f8e5-c486-4a42-abfd-a2fd5d7e8fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:07.145958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1359/2714] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:07.153934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42d4f-4768-4b43-bcda-1801078ebf09 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.154292 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42d4f-4768-4b43-bcda-1801078ebf09 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.154511 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42d4f-4768-4b43-bcda-1801078ebf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.180329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-7cb42d4f-4768-4b43-bcda-1801078ebf09 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:07.181699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-7cb42d4f-4768-4b43-bcda-1801078ebf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:07.191944 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1356/2715] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:07.196379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7f8e5-c486-4a42-abfd-a2fd5d7e8fd9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.206791 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7f8e5-c486-4a42-abfd-a2fd5d7e8fd9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.211627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.212445 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.212733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.213009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.213776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:07.214087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.215795 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.229694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1360/2716] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:28:07.234467 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42d4f-4768-4b43-bcda-1801078ebf09 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.244253 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42d4f-4768-4b43-bcda-1801078ebf09 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.249692 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd7151-e3e7-48bb-b784-11928427a77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.250369 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7151-e3e7-48bb-b784-11928427a77e admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.250651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7151-e3e7-48bb-b784-11928427a77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.251094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd7151-e3e7-48bb-b784-11928427a77e admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.251799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd7151-e3e7-48bb-b784-11928427a77e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.253830 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd7151-e3e7-48bb-b784-11928427a77e admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.287564 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1357/2717] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:07 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:07.291665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.304031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e46a04-8692-43b7-a9d0-3d58599dc9f4 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.377956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049bbe1-4570-4322-876e-9ab7e004744c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.378679 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049bbe1-4570-4322-876e-9ab7e004744c admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.378963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049bbe1-4570-4322-876e-9ab7e004744c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.379239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049bbe1-4570-4322-876e-9ab7e004744c admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.379955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049bbe1-4570-4322-876e-9ab7e004744c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.381925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049bbe1-4570-4322-876e-9ab7e004744c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.397492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:07.405405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1361/2718] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:07 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:07.409857 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd7151-e3e7-48bb-b784-11928427a77e admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.424044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd7151-e3e7-48bb-b784-11928427a77e admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.429615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80219fc8-41b5-427b-ae30-7331fe7df2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.430295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80219fc8-41b5-427b-ae30-7331fe7df2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.430605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80219fc8-41b5-427b-ae30-7331fe7df2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.430876 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80219fc8-41b5-427b-ae30-7331fe7df2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.431696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80219fc8-41b5-427b-ae30-7331fe7df2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.433429 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80219fc8-41b5-427b-ae30-7331fe7df2a7 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.445317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1358/2719] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:07.449987 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049bbe1-4570-4322-876e-9ab7e004744c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.460971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049bbe1-4570-4322-876e-9ab7e004744c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.468130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0491b18-8e6d-4557-a939-64e0e1d8d56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.468819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0491b18-8e6d-4557-a939-64e0e1d8d56c admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.469101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0491b18-8e6d-4557-a939-64e0e1d8d56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.469375 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0491b18-8e6d-4557-a939-64e0e1d8d56c admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.470151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0491b18-8e6d-4557-a939-64e0e1d8d56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.480294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0491b18-8e6d-4557-a939-64e0e1d8d56c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.483866 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1362/2720] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:07.488191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80219fc8-41b5-427b-ae30-7331fe7df2a7 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.499500 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80219fc8-41b5-427b-ae30-7331fe7df2a7 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.504072 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d01575-ce3a-4141-b5ac-6582ce3642e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.504958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d01575-ce3a-4141-b5ac-6582ce3642e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.505240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d01575-ce3a-4141-b5ac-6582ce3642e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.505516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d01575-ce3a-4141-b5ac-6582ce3642e4 admin admin] PATH_INFO: `/v3/projects/a00cc652ddb44eb1bbf03e0893773c44/users/db6f87de20704f1683a94d08845dfb0a/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.521174 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d01575-ce3a-4141-b5ac-6582ce3642e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a00cc652ddb44eb1bbf03e0893773c44, user_id=db6f87de20704f1683a94d08845dfb0a, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.523871 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d01575-ce3a-4141-b5ac-6582ce3642e4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.539288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1359/2721] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:07 2022] PUT /identity/v3/projects/a00cc652ddb44eb1bbf03e0893773c44/users/db6f87de20704f1683a94d08845dfb0a/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:07.550463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e130-5574-4aad-8a0f-9d9c4a777163 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.550767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e130-5574-4aad-8a0f-9d9c4a777163 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.551050 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24e130-5574-4aad-8a0f-9d9c4a777163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.582494 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7b24e130-5574-4aad-8a0f-9d9c4a777163 None None] MFA Rules not processed for user `db6f87de20704f1683a94d08845dfb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:07.583669 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7b24e130-5574-4aad-8a0f-9d9c4a777163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:07.663908 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1363/2722] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:07.669896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d01575-ce3a-4141-b5ac-6582ce3642e4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.744114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d01575-ce3a-4141-b5ac-6582ce3642e4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.756809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3293b9f-2ee3-4cbc-b98b-d306f2c4d367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.757493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293b9f-2ee3-4cbc-b98b-d306f2c4d367 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.757820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293b9f-2ee3-4cbc-b98b-d306f2c4d367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.758098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3293b9f-2ee3-4cbc-b98b-d306f2c4d367 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.758809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3293b9f-2ee3-4cbc-b98b-d306f2c4d367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.760750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3293b9f-2ee3-4cbc-b98b-d306f2c4d367 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.790660 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1360/2723] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:07 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:07.794930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e130-5574-4aad-8a0f-9d9c4a777163 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.805493 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24e130-5574-4aad-8a0f-9d9c4a777163 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.869508 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5381e0-da68-4921-b464-ed5ebd5d448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.870138 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5381e0-da68-4921-b464-ed5ebd5d448b admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.870455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5381e0-da68-4921-b464-ed5ebd5d448b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.870998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5381e0-da68-4921-b464-ed5ebd5d448b admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.872205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5381e0-da68-4921-b464-ed5ebd5d448b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.874194 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5381e0-da68-4921-b464-ed5ebd5d448b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.898410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1364/2724] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:07 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:07.904544 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3293b9f-2ee3-4cbc-b98b-d306f2c4d367 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.919993 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3293b9f-2ee3-4cbc-b98b-d306f2c4d367 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.927213 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59821ea-bf15-494d-a7aa-2c879e64fb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.927886 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59821ea-bf15-494d-a7aa-2c879e64fb2a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.928171 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59821ea-bf15-494d-a7aa-2c879e64fb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.928476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59821ea-bf15-494d-a7aa-2c879e64fb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.929205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59821ea-bf15-494d-a7aa-2c879e64fb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.930885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59821ea-bf15-494d-a7aa-2c879e64fb2a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:07.943089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1361/2725] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:07.950565 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5381e0-da68-4921-b464-ed5ebd5d448b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:07.965966 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5381e0-da68-4921-b464-ed5ebd5d448b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:07.972275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60339ab-a463-4ced-8f0e-835754e3fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:07.973187 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60339ab-a463-4ced-8f0e-835754e3fcfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:07.973509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60339ab-a463-4ced-8f0e-835754e3fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:07.973925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60339ab-a463-4ced-8f0e-835754e3fcfa admin admin] PATH_INFO: `/v3/projects/36c4fbc283b542f5b88d593366972dd6/users/2e53035e23e34247afc7cd63212a9b69/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:07.993585 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60339ab-a463-4ced-8f0e-835754e3fcfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c4fbc283b542f5b88d593366972dd6, user_id=2e53035e23e34247afc7cd63212a9b69, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:07.996545 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60339ab-a463-4ced-8f0e-835754e3fcfa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:08.011889 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1365/2726] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:07 2022] PUT /identity/v3/projects/36c4fbc283b542f5b88d593366972dd6/users/2e53035e23e34247afc7cd63212a9b69/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:08.016902 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59821ea-bf15-494d-a7aa-2c879e64fb2a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:08.027901 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59821ea-bf15-494d-a7aa-2c879e64fb2a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:08.063696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:08.103196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd43f6-f2a9-4721-a4d2-c8a5ef7f046b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:08.104090 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd43f6-f2a9-4721-a4d2-c8a5ef7f046b admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.104375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd43f6-f2a9-4721-a4d2-c8a5ef7f046b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.104647 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd43f6-f2a9-4721-a4d2-c8a5ef7f046b admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.106594 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd43f6-f2a9-4721-a4d2-c8a5ef7f046b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:08.108541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd43f6-f2a9-4721-a4d2-c8a5ef7f046b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:08.121954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1362/2727] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:08.124794 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60339ab-a463-4ced-8f0e-835754e3fcfa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:08.135797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60339ab-a463-4ced-8f0e-835754e3fcfa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:08.141915 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:08.142821 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.143102 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.143378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] PATH_INFO: `/v3/projects/36c4fbc283b542f5b88d593366972dd6/users/2e53035e23e34247afc7cd63212a9b69/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.145556 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36c4fbc283b542f5b88d593366972dd6, user_id=2e53035e23e34247afc7cd63212a9b69, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:08.148191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:08.160166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1366/2728] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:08 2022] PUT /identity/v3/projects/36c4fbc283b542f5b88d593366972dd6/users/2e53035e23e34247afc7cd63212a9b69/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:08.172221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd1f8-0a75-4133-87c6-fe3818fae92d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.172650 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd1f8-0a75-4133-87c6-fe3818fae92d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.173119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd1f8-0a75-4133-87c6-fe3818fae92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.209475 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-15afd1f8-0a75-4133-87c6-fe3818fae92d None None] MFA Rules not processed for user `2e53035e23e34247afc7cd63212a9b69`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:08.210759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-15afd1f8-0a75-4133-87c6-fe3818fae92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:08.282838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1363/2729] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:08.295332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:08.298213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:08.332234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:08.359554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a10fd-00f8-4a97-b6d8-b6771d0a9ef9 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:08.364339 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc6fb0-a88b-4afd-934b-b495e549ccd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:08.364805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc6fb0-a88b-4afd-934b-b495e549ccd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.365143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc6fb0-a88b-4afd-934b-b495e549ccd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.365496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc6fb0-a88b-4afd-934b-b495e549ccd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.377567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc6fb0-a88b-4afd-934b-b495e549ccd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:08.379223 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc6fb0-a88b-4afd-934b-b495e549ccd9 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:08.390489 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1367/2730] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:08.509295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd1f8-0a75-4133-87c6-fe3818fae92d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:08.510795 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-15afd1f8-0a75-4133-87c6-fe3818fae92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:08.580456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd1f8-0a75-4133-87c6-fe3818fae92d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:08.585442 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:08.586354 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.586636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.586907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] PATH_INFO: `/v3/users/fc989f2fe18346fba53a08559e53d4b1` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.588087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc989f2fe18346fba53a08559e53d4b1)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:08.590605 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:08.643787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc989f2fe18346fba53a08559e53d4b1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:08.647851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:08.652278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc989f2fe18346fba53a08559e53d4b1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:08.669534 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc989f2fe18346fba53a08559e53d4b1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:08.670288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc989f2fe18346fba53a08559e53d4b1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:08.670900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc989f2fe18346fba53a08559e53d4b1'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:08.684954 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1364/2731] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:08 2022] DELETE /identity/v3/users/fc989f2fe18346fba53a08559e53d4b1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:08.690211 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13475da-9435-415e-8473-5c055436c5af None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.690595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13475da-9435-415e-8473-5c055436c5af None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.690924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13475da-9435-415e-8473-5c055436c5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.717644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-d13475da-9435-415e-8473-5c055436c5af None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:08.717644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d13475da-9435-415e-8473-5c055436c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:08.747875 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:08.750280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:08.783320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1368/2732] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:08.795933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13475da-9435-415e-8473-5c055436c5af None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:08.797463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d13475da-9435-415e-8473-5c055436c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:08.846943 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622359b-3ff8-44ac-bc3d-bae729ce5fa1 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:08.852395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e53035e23e34247afc7cd63212a9b69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36c4fbc283b542f5b88d593366972dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:08.853239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.853537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.854279 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.855155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:08.855481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:08.857409 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:08.882940 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1365/2733] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:28:08.884809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:08.891014 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13475da-9435-415e-8473-5c055436c5af None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:08.896847 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35410f-3377-42e6-9e05-ec369a133f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:08.897747 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35410f-3377-42e6-9e05-ec369a133f52 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.898161 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35410f-3377-42e6-9e05-ec369a133f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.898554 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35410f-3377-42e6-9e05-ec369a133f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.900247 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c5144a-c34e-4a64-aa75-042ad3cd3560 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:08.907590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb7f07-bf4e-453c-a32c-3ed2f69ff392 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e53035e23e34247afc7cd63212a9b69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36c4fbc283b542f5b88d593366972dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:08.908328 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7f07-bf4e-453c-a32c-3ed2f69ff392 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:08.908617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7f07-bf4e-453c-a32c-3ed2f69ff392 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:08.908903 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fb7f07-bf4e-453c-a32c-3ed2f69ff392 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:08.909600 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb7f07-bf4e-453c-a32c-3ed2f69ff392 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:08.912248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fb7f07-bf4e-453c-a32c-3ed2f69ff392 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:08.914978 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35410f-3377-42e6-9e05-ec369a133f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:08.916746 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35410f-3377-42e6-9e05-ec369a133f52 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:08.927845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1369/2734] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:28:08.949116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1366/2735] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:08 2022] POST /identity/v3/projects => generated 354 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:08.950429 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35410f-3377-42e6-9e05-ec369a133f52 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:08.961763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35410f-3377-42e6-9e05-ec369a133f52 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.038202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e53035e23e34247afc7cd63212a9b69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36c4fbc283b542f5b88d593366972dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.039075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.039436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.039786 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.040609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:09.041278 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.043824 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.059442 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1370/2736] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:28:09.097438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb7f07-bf4e-453c-a32c-3ed2f69ff392 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.113595 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fb7f07-bf4e-453c-a32c-3ed2f69ff392 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.120927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e53035e23e34247afc7cd63212a9b69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '36c4fbc283b542f5b88d593366972dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.123827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.123827 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.124295 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] PATH_INFO: `/v3/projects/91e12b47b2c14e4590ac452b5edabd6d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.126110 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Authorizing `identity:delete_project(project_id=91e12b47b2c14e4590ac452b5edabd6d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.131242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.132266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.134426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:09.164921 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:09.192578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e12b47b2c14e4590ac452b5edabd6d', 'request_id': 'req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.199338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e12b47b2c14e4590ac452b5edabd6d', 'request_id': 'req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.207906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e12b47b2c14e4590ac452b5edabd6d', 'request_id': 'req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.219767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] The token cache is being invalidate because project 91e12b47b2c14e4590ac452b5edabd6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:09.220060 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.220718 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-59fab678-5a0b-492b-a336-5ef8b5ca9b8f tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.222342 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1367/2737] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:09 2022] DELETE /identity/v3/projects/91e12b47b2c14e4590ac452b5edabd6d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:09.230821 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecaab30-aa63-486a-84da-30ae6b494728 tempest-VolumeQuotaClassesTest-578716324 tempest-VolumeQuotaClassesTest-578716324-project-admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.294170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.295260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.295549 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.296151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] PATH_INFO: `/v3/projects/d944af7f74394542ac7b6cb64beb6070` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.297113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d944af7f74394542ac7b6cb64beb6070)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.299166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.351928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd944af7f74394542ac7b6cb64beb6070', 'request_id': 'req-68255005-46bf-4b7b-b56e-c14752bdfdf1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.361131 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd944af7f74394542ac7b6cb64beb6070', 'request_id': 'req-68255005-46bf-4b7b-b56e-c14752bdfdf1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.368823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd944af7f74394542ac7b6cb64beb6070', 'request_id': 'req-68255005-46bf-4b7b-b56e-c14752bdfdf1'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.381285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] The token cache is being invalidate because project d944af7f74394542ac7b6cb64beb6070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:09.381505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.382105 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.387986 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1371/2738] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:09 2022] DELETE /identity/v3/projects/d944af7f74394542ac7b6cb64beb6070 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:09.390029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae164d40-8ce2-446d-b0d8-c99a0a83cb89 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.390648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae164d40-8ce2-446d-b0d8-c99a0a83cb89 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.391138 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae164d40-8ce2-446d-b0d8-c99a0a83cb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.414768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-ae164d40-8ce2-446d-b0d8-c99a0a83cb89 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:09.416069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ae164d40-8ce2-446d-b0d8-c99a0a83cb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:09.442198 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:09.443386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.444979 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:09.490728 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1368/2739] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:09.497917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:09.500353 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30474703-e781-480a-abe2-daaa5fb5ec4d None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.500679 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30474703-e781-480a-abe2-daaa5fb5ec4d None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.500963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30474703-e781-480a-abe2-daaa5fb5ec4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.518566 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68255005-46bf-4b7b-b56e-c14752bdfdf1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.520272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-30474703-e781-480a-abe2-daaa5fb5ec4d None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:09.521707 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-30474703-e781-480a-abe2-daaa5fb5ec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:09.523702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.524599 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.524880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.525159 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] PATH_INFO: `/v3/users/db6f87de20704f1683a94d08845dfb0a` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.526592 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db6f87de20704f1683a94d08845dfb0a)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.529385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.530612 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1369/2740] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:09.535059 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30474703-e781-480a-abe2-daaa5fb5ec4d None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.544366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30474703-e781-480a-abe2-daaa5fb5ec4d None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.549505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.550406 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.550727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.550998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.551714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:09.552013 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.553616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.566394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1370/2741] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:28:09.570908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.581883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604a5a8-67ab-436c-bfd1-7024ea406d51 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.585439 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db6f87de20704f1683a94d08845dfb0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.586889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.587539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.587818 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.588093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.588781 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.591048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.595218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db6f87de20704f1683a94d08845dfb0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.609216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db6f87de20704f1683a94d08845dfb0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.610293 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db6f87de20704f1683a94d08845dfb0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.611243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5b677d05-59fc-4cb0-ac97-4478af41bad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db6f87de20704f1683a94d08845dfb0a'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:09.619784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1371/2742] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:09 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:09.624201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.624535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1372/2743] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:09 2022] DELETE /identity/v3/users/db6f87de20704f1683a94d08845dfb0a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:09.626081 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:09.628778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5398c0c-f17e-44f4-83d9-9d63d8fb6ddb None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.629071 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5398c0c-f17e-44f4-83d9-9d63d8fb6ddb None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.629367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5398c0c-f17e-44f4-83d9-9d63d8fb6ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.647974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a5398c0c-f17e-44f4-83d9-9d63d8fb6ddb None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:09.649189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a5398c0c-f17e-44f4-83d9-9d63d8fb6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:09.698994 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593f283-9402-4280-bc0a-94f332371014 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.712009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1373/2744] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:09.722081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5398c0c-f17e-44f4-83d9-9d63d8fb6ddb None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.724085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a5398c0c-f17e-44f4-83d9-9d63d8fb6ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:09.758949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281239d-ae9c-4c18-9630-f3fe0fc09f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.759726 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281239d-ae9c-4c18-9630-f3fe0fc09f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.760041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281239d-ae9c-4c18-9630-f3fe0fc09f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.760335 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281239d-ae9c-4c18-9630-f3fe0fc09f0c admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.761088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281239d-ae9c-4c18-9630-f3fe0fc09f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.763172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281239d-ae9c-4c18-9630-f3fe0fc09f0c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.787014 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1372/2745] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:09 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:09.787263 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5398c0c-f17e-44f4-83d9-9d63d8fb6ddb None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.788250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281239d-ae9c-4c18-9630-f3fe0fc09f0c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.790541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efb5bf-edfa-44e8-aded-778349f650aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.791198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efb5bf-edfa-44e8-aded-778349f650aa service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.791480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efb5bf-edfa-44e8-aded-778349f650aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.791752 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efb5bf-edfa-44e8-aded-778349f650aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.798336 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281239d-ae9c-4c18-9630-f3fe0fc09f0c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.801893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0efb5bf-edfa-44e8-aded-778349f650aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.803228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3706146-2161-4b67-96c5-40a53b32910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.803366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0efb5bf-edfa-44e8-aded-778349f650aa service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.803885 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3706146-2161-4b67-96c5-40a53b32910a admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.804169 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3706146-2161-4b67-96c5-40a53b32910a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.804444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3706146-2161-4b67-96c5-40a53b32910a admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.805172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3706146-2161-4b67-96c5-40a53b32910a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.806808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3706146-2161-4b67-96c5-40a53b32910a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.813537 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1374/2746] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:28:09.820704 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1373/2747] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:09.822179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efb5bf-edfa-44e8-aded-778349f650aa service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.832465 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efb5bf-edfa-44e8-aded-778349f650aa service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.837555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b2ff85-45c1-47c5-9df2-9313d28c4afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.838343 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2ff85-45c1-47c5-9df2-9313d28c4afa admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.838627 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2ff85-45c1-47c5-9df2-9313d28c4afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.838903 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b2ff85-45c1-47c5-9df2-9313d28c4afa admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.840031 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2ff85-45c1-47c5-9df2-9313d28c4afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.841765 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b2ff85-45c1-47c5-9df2-9313d28c4afa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.844341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:09.859662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1375/2748] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:09.860923 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3706146-2161-4b67-96c5-40a53b32910a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:09.873682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3706146-2161-4b67-96c5-40a53b32910a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:09.879625 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:09.880898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.881294 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.881629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] PATH_INFO: `/v3/projects/bfe924f59fd34f58b443e1c324dc581c/users/0b41303f85c7455087c8d25658f1a548/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.886117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:09.903000 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe924f59fd34f58b443e1c324dc581c, user_id=0b41303f85c7455087c8d25658f1a548, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:09.906301 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:09.922389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1374/2749] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:09 2022] PUT /identity/v3/projects/bfe924f59fd34f58b443e1c324dc581c/users/0b41303f85c7455087c8d25658f1a548/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:09.927583 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c746-9d54-488a-8ef6-0035362c0d55 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:09.927875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c746-9d54-488a-8ef6-0035362c0d55 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:09.928152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1c746-9d54-488a-8ef6-0035362c0d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:09.947377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-bfd1c746-9d54-488a-8ef6-0035362c0d55 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:09.948839 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bfd1c746-9d54-488a-8ef6-0035362c0d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:10.031022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1376/2750] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:10.041898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:10.043850 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:10.118313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01a2a9-a6e0-435d-9600-0de562818337 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:10.125117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946cbae-aba4-4a31-b149-1f568edc2616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:10.125117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cbae-aba4-4a31-b149-1f568edc2616 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:10.125117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cbae-aba4-4a31-b149-1f568edc2616 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:10.125117 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946cbae-aba4-4a31-b149-1f568edc2616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:10.127620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1c746-9d54-488a-8ef6-0035362c0d55 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:10.129852 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-bfd1c746-9d54-488a-8ef6-0035362c0d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:10.138955 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946cbae-aba4-4a31-b149-1f568edc2616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:10.140642 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946cbae-aba4-4a31-b149-1f568edc2616 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:10.152678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1c746-9d54-488a-8ef6-0035362c0d55 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:10.153303 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1375/2751] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:10.159484 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:10.160382 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:10.160658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:10.160928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] PATH_INFO: `/v3/projects/a00cc652ddb44eb1bbf03e0893773c44` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:10.162255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=a00cc652ddb44eb1bbf03e0893773c44)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:10.164452 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:10.248403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a00cc652ddb44eb1bbf03e0893773c44', 'request_id': 'req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.272076 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a00cc652ddb44eb1bbf03e0893773c44', 'request_id': 'req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.283391 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a00cc652ddb44eb1bbf03e0893773c44', 'request_id': 'req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.296496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] The token cache is being invalidate because project a00cc652ddb44eb1bbf03e0893773c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:10.297064 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.297924 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.309878 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1377/2752] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:10 2022] DELETE /identity/v3/projects/a00cc652ddb44eb1bbf03e0893773c44 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:10.309984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946cbae-aba4-4a31-b149-1f568edc2616 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:10.316856 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946cbae-aba4-4a31-b149-1f568edc2616 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:10.390287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:10.391333 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:10.391683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:10.392075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] PATH_INFO: `/v3/users/2e53035e23e34247afc7cd63212a9b69` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:10.393272 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e53035e23e34247afc7cd63212a9b69)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:10.395239 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:10.448770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e53035e23e34247afc7cd63212a9b69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.458682 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e53035e23e34247afc7cd63212a9b69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.474628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e53035e23e34247afc7cd63212a9b69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.475868 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e53035e23e34247afc7cd63212a9b69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.476982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d086f89-1380-478e-8612-aedb9a6c2d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e53035e23e34247afc7cd63212a9b69'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:10.494144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1376/2753] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:10 2022] DELETE /identity/v3/users/2e53035e23e34247afc7cd63212a9b69 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:10.952038 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:10.953541 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:10.974899 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f861dda-7768-48c7-8ccd-f42ecd9a4ada admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:11.047450 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:11.048481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.048767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.049851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] PATH_INFO: `/v3/projects/36c4fbc283b542f5b88d593366972dd6` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.052358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36c4fbc283b542f5b88d593366972dd6)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:11.054905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:11.091753 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:11.121737 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36c4fbc283b542f5b88d593366972dd6', 'request_id': 'req-02a0bd67-963c-4d26-a265-28d74354f6e3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:11.132368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36c4fbc283b542f5b88d593366972dd6', 'request_id': 'req-02a0bd67-963c-4d26-a265-28d74354f6e3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:11.141517 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36c4fbc283b542f5b88d593366972dd6', 'request_id': 'req-02a0bd67-963c-4d26-a265-28d74354f6e3'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:11.155609 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] The token cache is being invalidate because project 36c4fbc283b542f5b88d593366972dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:11.155930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:11.156618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-02a0bd67-963c-4d26-a265-28d74354f6e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:11.158577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1378/2754] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:10 2022] DELETE /identity/v3/projects/36c4fbc283b542f5b88d593366972dd6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:11.163639 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1594c2b-4e61-4727-a8a1-47542c19cc80 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.164182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1594c2b-4e61-4727-a8a1-47542c19cc80 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.164182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1594c2b-4e61-4727-a8a1-47542c19cc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.177123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:11.184532 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-c1594c2b-4e61-4727-a8a1-47542c19cc80 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:11.186783 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c1594c2b-4e61-4727-a8a1-47542c19cc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:11.261816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1377/2755] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:11.271818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007f5e3-c9d7-4acc-b0c5-704dde5ce9e9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.272184 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007f5e3-c9d7-4acc-b0c5-704dde5ce9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.272643 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007f5e3-c9d7-4acc-b0c5-704dde5ce9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.294097 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-7007f5e3-c9d7-4acc-b0c5-704dde5ce9e9 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:11.295614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-7007f5e3-c9d7-4acc-b0c5-704dde5ce9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:11.304017 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1379/2756] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:11.308471 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1594c2b-4e61-4727-a8a1-47542c19cc80 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:11.320812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1594c2b-4e61-4727-a8a1-47542c19cc80 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:11.326899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:11.327751 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.328141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.328521 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.329370 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:11.329832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:11.331694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:11.349502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1378/2757] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:28:11.350857 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007f5e3-c9d7-4acc-b0c5-704dde5ce9e9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:11.362200 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007f5e3-c9d7-4acc-b0c5-704dde5ce9e9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:11.379940 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e095e-7c65-44da-b971-29d1852b4b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:11.381205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e095e-7c65-44da-b971-29d1852b4b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.381538 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e095e-7c65-44da-b971-29d1852b4b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.381902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e095e-7c65-44da-b971-29d1852b4b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.382620 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e095e-7c65-44da-b971-29d1852b4b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:11.384923 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e095e-7c65-44da-b971-29d1852b4b9d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:11.434733 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1380/2758] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:11 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:11.438769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:11.454910 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4c0e8-595e-43e3-b888-0df8f19e6dd4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:11.545219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a7c09-78b4-4cd7-af71-ee7b2955e16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:11.546315 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a7c09-78b4-4cd7-af71-ee7b2955e16a admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.547186 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a7c09-78b4-4cd7-af71-ee7b2955e16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.547574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a7c09-78b4-4cd7-af71-ee7b2955e16a admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.548502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a7c09-78b4-4cd7-af71-ee7b2955e16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:11.551049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a7c09-78b4-4cd7-af71-ee7b2955e16a admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:11.587870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1379/2759] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:11 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:11.594507 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e095e-7c65-44da-b971-29d1852b4b9d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:11.610408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e095e-7c65-44da-b971-29d1852b4b9d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:11.618415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb32ff-0d16-4baa-99da-4adac8faba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:11.619661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb32ff-0d16-4baa-99da-4adac8faba4e admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.620160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb32ff-0d16-4baa-99da-4adac8faba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.620784 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb32ff-0d16-4baa-99da-4adac8faba4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.622229 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb32ff-0d16-4baa-99da-4adac8faba4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:11.625265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb32ff-0d16-4baa-99da-4adac8faba4e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:11.662759 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1381/2760] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:11.667879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a7c09-78b4-4cd7-af71-ee7b2955e16a admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:11.685626 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a7c09-78b4-4cd7-af71-ee7b2955e16a admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:11.695779 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4355d6f-2af0-4c57-a7fb-f754cf518340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:11.696325 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4355d6f-2af0-4c57-a7fb-f754cf518340 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.696665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4355d6f-2af0-4c57-a7fb-f754cf518340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.696950 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4355d6f-2af0-4c57-a7fb-f754cf518340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.697881 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4355d6f-2af0-4c57-a7fb-f754cf518340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:11.700574 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4355d6f-2af0-4c57-a7fb-f754cf518340 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:11.717152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1380/2761] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:11.721020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb32ff-0d16-4baa-99da-4adac8faba4e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:11.735642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb32ff-0d16-4baa-99da-4adac8faba4e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:11.747828 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fea14b-c5df-4baf-9514-274294cd3e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:11.750272 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fea14b-c5df-4baf-9514-274294cd3e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.751718 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fea14b-c5df-4baf-9514-274294cd3e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.753434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fea14b-c5df-4baf-9514-274294cd3e7e admin admin] PATH_INFO: `/v3/projects/81bf128e2d1c4e1ab8f3b47a54141c4d/users/d48a7828a78b4339b9df88a0bfcd58d2/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.778294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fea14b-c5df-4baf-9514-274294cd3e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bf128e2d1c4e1ab8f3b47a54141c4d, user_id=d48a7828a78b4339b9df88a0bfcd58d2, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:11.781340 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fea14b-c5df-4baf-9514-274294cd3e7e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:11.818359 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60955c-708b-4c7c-b925-2cdefa2cf681 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:11.818864 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60955c-708b-4c7c-b925-2cdefa2cf681 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:11.819401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d60955c-708b-4c7c-b925-2cdefa2cf681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:11.826904 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1382/2762] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:11 2022] PUT /identity/v3/projects/81bf128e2d1c4e1ab8f3b47a54141c4d/users/d48a7828a78b4339b9df88a0bfcd58d2/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:11.877845 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-1d60955c-708b-4c7c-b925-2cdefa2cf681 None None] MFA Rules not processed for user `d48a7828a78b4339b9df88a0bfcd58d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:11.884201 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-1d60955c-708b-4c7c-b925-2cdefa2cf681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:11.981742 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1381/2763] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:11.984174 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fea14b-c5df-4baf-9514-274294cd3e7e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:12.073095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fea14b-c5df-4baf-9514-274294cd3e7e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:12.078414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea2bbb-d2e8-4122-b2b1-6da8c4f99762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:12.079127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea2bbb-d2e8-4122-b2b1-6da8c4f99762 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.079415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea2bbb-d2e8-4122-b2b1-6da8c4f99762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.079691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea2bbb-d2e8-4122-b2b1-6da8c4f99762 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:12.080386 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea2bbb-d2e8-4122-b2b1-6da8c4f99762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:12.082385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea2bbb-d2e8-4122-b2b1-6da8c4f99762 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:12.124551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1383/2764] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:11 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:12.126329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60955c-708b-4c7c-b925-2cdefa2cf681 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:12.132437 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:12.145070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d60955c-708b-4c7c-b925-2cdefa2cf681 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:12.224384 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce930d-a2fe-4896-b9aa-36ea98eede94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:12.225076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce930d-a2fe-4896-b9aa-36ea98eede94 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.225458 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce930d-a2fe-4896-b9aa-36ea98eede94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.226020 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce930d-a2fe-4896-b9aa-36ea98eede94 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:12.226741 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce930d-a2fe-4896-b9aa-36ea98eede94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:12.228847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce930d-a2fe-4896-b9aa-36ea98eede94 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:12.253049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1382/2765] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:12 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:12.254327 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea2bbb-d2e8-4122-b2b1-6da8c4f99762 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:12.265427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea2bbb-d2e8-4122-b2b1-6da8c4f99762 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:12.270209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c5f61-c9db-42b5-812f-ef5fa9889b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:12.270926 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5f61-c9db-42b5-812f-ef5fa9889b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.271244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5f61-c9db-42b5-812f-ef5fa9889b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.271522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c5f61-c9db-42b5-812f-ef5fa9889b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:12.272681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c5f61-c9db-42b5-812f-ef5fa9889b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:12.273989 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c5f61-c9db-42b5-812f-ef5fa9889b63 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:12.286310 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1384/2766] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:12.290699 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce930d-a2fe-4896-b9aa-36ea98eede94 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:12.301305 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce930d-a2fe-4896-b9aa-36ea98eede94 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:12.306597 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e063a7-efe8-4bec-927b-281374fdaf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:12.307421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e063a7-efe8-4bec-927b-281374fdaf82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.307709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e063a7-efe8-4bec-927b-281374fdaf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.308147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e063a7-efe8-4bec-927b-281374fdaf82 admin admin] PATH_INFO: `/v3/projects/cb2409ad083f443390792810caaa142f/users/65bb5ea4af1049fb8748c63031209949/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:12.325703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e063a7-efe8-4bec-927b-281374fdaf82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2409ad083f443390792810caaa142f, user_id=65bb5ea4af1049fb8748c63031209949, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:12.328235 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e063a7-efe8-4bec-927b-281374fdaf82 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:12.343391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1383/2767] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:12 2022] PUT /identity/v3/projects/cb2409ad083f443390792810caaa142f/users/65bb5ea4af1049fb8748c63031209949/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:12.347261 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c5f61-c9db-42b5-812f-ef5fa9889b63 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:12.359168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c5f61-c9db-42b5-812f-ef5fa9889b63 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:12.383558 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:12.445008 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24406624-e42e-4a2f-b53a-72f454cb92c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:12.445767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24406624-e42e-4a2f-b53a-72f454cb92c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.446054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24406624-e42e-4a2f-b53a-72f454cb92c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.446358 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24406624-e42e-4a2f-b53a-72f454cb92c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:12.447052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24406624-e42e-4a2f-b53a-72f454cb92c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:12.448711 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24406624-e42e-4a2f-b53a-72f454cb92c0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:12.459520 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1385/2768] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:12.463938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e063a7-efe8-4bec-927b-281374fdaf82 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:12.480094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e063a7-efe8-4bec-927b-281374fdaf82 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:12.487542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:12.489266 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.489810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.490464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] PATH_INFO: `/v3/projects/cb2409ad083f443390792810caaa142f/users/65bb5ea4af1049fb8748c63031209949/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:12.493206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2409ad083f443390792810caaa142f, user_id=65bb5ea4af1049fb8748c63031209949, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:12.496791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:12.510674 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1384/2769] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:12 2022] PUT /identity/v3/projects/cb2409ad083f443390792810caaa142f/users/65bb5ea4af1049fb8748c63031209949/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:12.520927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0b4ac-eaad-405c-ae0a-13dd5b1121a3 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.521219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0b4ac-eaad-405c-ae0a-13dd5b1121a3 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.521649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0b4ac-eaad-405c-ae0a-13dd5b1121a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:12.564367 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1bb0b4ac-eaad-405c-ae0a-13dd5b1121a3 None None] MFA Rules not processed for user `65bb5ea4af1049fb8748c63031209949`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:12.566018 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1bb0b4ac-eaad-405c-ae0a-13dd5b1121a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:12.684731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1386/2770] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:12.696943 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:12.700195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:12.790793 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec810b-d8f2-4826-a77f-ae55626244db admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:12.805307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c33768-067d-45fa-9515-0785bcd2f2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:12.806030 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c33768-067d-45fa-9515-0785bcd2f2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.806508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c33768-067d-45fa-9515-0785bcd2f2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.806629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c33768-067d-45fa-9515-0785bcd2f2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:12.825584 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c33768-067d-45fa-9515-0785bcd2f2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:12.827109 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c33768-067d-45fa-9515-0785bcd2f2a7 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:12.841473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1385/2771] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:12.952613 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0b4ac-eaad-405c-ae0a-13dd5b1121a3 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:12.954146 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1bb0b4ac-eaad-405c-ae0a-13dd5b1121a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:12.974323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0b4ac-eaad-405c-ae0a-13dd5b1121a3 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:12.985818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a327180-d2e6-4dce-9242-5fa78e8ab052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:12.986504 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327180-d2e6-4dce-9242-5fa78e8ab052 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:12.986789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327180-d2e6-4dce-9242-5fa78e8ab052 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:12.987081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a327180-d2e6-4dce-9242-5fa78e8ab052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:13.077559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a327180-d2e6-4dce-9242-5fa78e8ab052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:13.079106 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a327180-d2e6-4dce-9242-5fa78e8ab052 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:13.089642 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1387/2772] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:13.442008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c33768-067d-45fa-9515-0785bcd2f2a7 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:13.523967 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:13.530400 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c33768-067d-45fa-9515-0785bcd2f2a7 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:13.538062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:13.539131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:13.539632 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:13.540062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] PATH_INFO: `/v3/users/d48a7828a78b4339b9df88a0bfcd58d2` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:13.541496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d48a7828a78b4339b9df88a0bfcd58d2)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:13.543838 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:13.614018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd48a7828a78b4339b9df88a0bfcd58d2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:13.624787 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd48a7828a78b4339b9df88a0bfcd58d2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:13.657193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd48a7828a78b4339b9df88a0bfcd58d2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:13.658263 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd48a7828a78b4339b9df88a0bfcd58d2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:13.659148 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd48a7828a78b4339b9df88a0bfcd58d2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:13.676545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1386/2773] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:13 2022] DELETE /identity/v3/users/d48a7828a78b4339b9df88a0bfcd58d2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:13.681334 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36fcd9-4363-4fff-9af6-464f60851ce8 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:13.681778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36fcd9-4363-4fff-9af6-464f60851ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:13.684835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36fcd9-4363-4fff-9af6-464f60851ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:13.709571 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-9e36fcd9-4363-4fff-9af6-464f60851ce8 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:13.712734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9e36fcd9-4363-4fff-9af6-464f60851ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:13.767183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:13.798576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1388/2774] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:13.808816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:13.810588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:13.911393 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7b4a5-0a17-41f9-89f7-fe3562e74a5b admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:13.918629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f81d56-f03f-4121-a837-cfd3f4c395ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:13.919464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81d56-f03f-4121-a837-cfd3f4c395ef service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:13.919907 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81d56-f03f-4121-a837-cfd3f4c395ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:13.920284 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f81d56-f03f-4121-a837-cfd3f4c395ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:13.932738 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f81d56-f03f-4121-a837-cfd3f4c395ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:13.935904 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f81d56-f03f-4121-a837-cfd3f4c395ef service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:13.971221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1387/2775] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:14.368268 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36fcd9-4363-4fff-9af6-464f60851ce8 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:14.375415 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9e36fcd9-4363-4fff-9af6-464f60851ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:14.412052 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36fcd9-4363-4fff-9af6-464f60851ce8 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:14.419509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:14.420434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:14.420681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:14.420953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] PATH_INFO: `/v3/projects/81bf128e2d1c4e1ab8f3b47a54141c4d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:14.422454 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bf128e2d1c4e1ab8f3b47a54141c4d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:14.425453 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:14.508142 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bf128e2d1c4e1ab8f3b47a54141c4d', 'request_id': 'req-e3136055-8f13-4c92-b348-39ea2346b39d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.528081 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bf128e2d1c4e1ab8f3b47a54141c4d', 'request_id': 'req-e3136055-8f13-4c92-b348-39ea2346b39d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.545044 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bf128e2d1c4e1ab8f3b47a54141c4d', 'request_id': 'req-e3136055-8f13-4c92-b348-39ea2346b39d'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.559870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] The token cache is being invalidate because project 81bf128e2d1c4e1ab8f3b47a54141c4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:14.560266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.561036 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-e3136055-8f13-4c92-b348-39ea2346b39d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.562662 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1389/2776] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:14 2022] DELETE /identity/v3/projects/81bf128e2d1c4e1ab8f3b47a54141c4d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:14.567241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f81d56-f03f-4121-a837-cfd3f4c395ef service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:14.580748 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f81d56-f03f-4121-a837-cfd3f4c395ef service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:14.683975 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:14.684943 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:14.685232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:14.685512 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] PATH_INFO: `/v3/users/65bb5ea4af1049fb8748c63031209949` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:14.688216 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] RBAC: Authorizing `identity:delete_user(user_id=65bb5ea4af1049fb8748c63031209949)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:14.690260 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:14.732079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:14.806432 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65bb5ea4af1049fb8748c63031209949'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.816807 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65bb5ea4af1049fb8748c63031209949'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.833132 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65bb5ea4af1049fb8748c63031209949'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.833924 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65bb5ea4af1049fb8748c63031209949'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.834520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65bb5ea4af1049fb8748c63031209949'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:14.850645 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e70423-61e3-4db6-a3ff-50bbefe6aa65 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:14.851206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e70423-61e3-4db6-a3ff-50bbefe6aa65 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:14.851206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e70423-61e3-4db6-a3ff-50bbefe6aa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:14.856046 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1388/2777] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:14 2022] DELETE /identity/v3/users/65bb5ea4af1049fb8748c63031209949 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:14.896579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-a8e70423-61e3-4db6-a3ff-50bbefe6aa65 None None] MFA Rules not processed for user `0b41303f85c7455087c8d25658f1a548`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:14.897974 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a8e70423-61e3-4db6-a3ff-50bbefe6aa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:14.987221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1390/2778] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:14.992340 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:14.993866 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:15.083164 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c6f81-7134-4917-ac4d-9d6fc1fbe4df admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.097614 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.098396 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.098684 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.098961 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.099669 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.101844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.137492 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1389/2779] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:14 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:15.147062 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e70423-61e3-4db6-a3ff-50bbefe6aa65 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.159618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e70423-61e3-4db6-a3ff-50bbefe6aa65 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.216920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.220844 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:15.221774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:15.237468 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1521c-e035-4249-ab94-780ef381e3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.238851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1521c-e035-4249-ab94-780ef381e3ea admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.239414 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1521c-e035-4249-ab94-780ef381e3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.239818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1521c-e035-4249-ab94-780ef381e3ea admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.241213 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1521c-e035-4249-ab94-780ef381e3ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.245925 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1521c-e035-4249-ab94-780ef381e3ea admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.271885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1391/2780] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:15 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:15.277163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1521c-e035-4249-ab94-780ef381e3ea admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.288526 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1521c-e035-4249-ab94-780ef381e3ea admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.295302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda4232-f35f-4d9b-8c31-3c57ba14f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.296424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda4232-f35f-4d9b-8c31-3c57ba14f865 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.296897 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda4232-f35f-4d9b-8c31-3c57ba14f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.297614 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda4232-f35f-4d9b-8c31-3c57ba14f865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.298766 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda4232-f35f-4d9b-8c31-3c57ba14f865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.301744 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda4232-f35f-4d9b-8c31-3c57ba14f865 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.316101 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1392/2781] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:15.320611 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda4232-f35f-4d9b-8c31-3c57ba14f865 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.334003 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda4232-f35f-4d9b-8c31-3c57ba14f865 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.337619 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bc525-6d23-4883-b872-0d6659ceb381 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.340368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90528d-8232-40ff-89fe-b94e31dd59ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.341245 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90528d-8232-40ff-89fe-b94e31dd59ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.341540 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90528d-8232-40ff-89fe-b94e31dd59ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.341851 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90528d-8232-40ff-89fe-b94e31dd59ae admin admin] PATH_INFO: `/v3/projects/b99b488d577a4f9098769a745789f207/users/60effcfa14064f16aa7f98e286da3e42/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.343095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.344017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.344307 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.346087 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] PATH_INFO: `/v3/projects/cb2409ad083f443390792810caaa142f` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.347491 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2409ad083f443390792810caaa142f)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.349456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.370061 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90528d-8232-40ff-89fe-b94e31dd59ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99b488d577a4f9098769a745789f207, user_id=60effcfa14064f16aa7f98e286da3e42, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.373593 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90528d-8232-40ff-89fe-b94e31dd59ae admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.384908 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:15.389912 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1393/2782] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:15 2022] PUT /identity/v3/projects/b99b488d577a4f9098769a745789f207/users/60effcfa14064f16aa7f98e286da3e42/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:15.394046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90528d-8232-40ff-89fe-b94e31dd59ae admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.404546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2409ad083f443390792810caaa142f', 'request_id': 'req-8d8a0834-c463-4f50-995e-b8ef1ebd008c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:15.411338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2409ad083f443390792810caaa142f', 'request_id': 'req-8d8a0834-c463-4f50-995e-b8ef1ebd008c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:15.420884 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2409ad083f443390792810caaa142f', 'request_id': 'req-8d8a0834-c463-4f50-995e-b8ef1ebd008c'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:15.431777 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] The token cache is being invalidate because project cb2409ad083f443390792810caaa142f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:15.432071 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:15.432821 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8d8a0834-c463-4f50-995e-b8ef1ebd008c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:15.434563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1390/2783] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:15 2022] DELETE /identity/v3/projects/cb2409ad083f443390792810caaa142f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:15.439770 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96d05-a7bf-40f9-9a99-9289b9e47389 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.440060 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96d05-a7bf-40f9-9a99-9289b9e47389 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.440338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96d05-a7bf-40f9-9a99-9289b9e47389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.458670 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-9ec96d05-a7bf-40f9-9a99-9289b9e47389 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:15.459880 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-9ec96d05-a7bf-40f9-9a99-9289b9e47389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:15.463298 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90528d-8232-40ff-89fe-b94e31dd59ae admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.524119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1391/2784] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:15.526301 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.527690 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.527970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.528254 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.529027 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.530704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.533405 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ae1c9-9822-411a-80e8-016d503d20df None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.533727 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ae1c9-9822-411a-80e8-016d503d20df None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.534011 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ae1c9-9822-411a-80e8-016d503d20df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.541594 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1394/2785] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:15.545809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.547639 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:15.555784 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-df4ae1c9-9822-411a-80e8-016d503d20df None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:15.556992 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-df4ae1c9-9822-411a-80e8-016d503d20df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:15.561834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f21a3a-7e6d-4f6f-98d1-c837559babdd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.570062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1392/2786] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:15.570598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd7542-ebce-4c4a-ad8f-408ebd8c62cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.570598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd7542-ebce-4c4a-ad8f-408ebd8c62cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.570598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd7542-ebce-4c4a-ad8f-408ebd8c62cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.570598 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd7542-ebce-4c4a-ad8f-408ebd8c62cd admin admin] PATH_INFO: `/v3/projects/b99b488d577a4f9098769a745789f207/users/60effcfa14064f16aa7f98e286da3e42/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.571278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ae1c9-9822-411a-80e8-016d503d20df None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.571433 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd7542-ebce-4c4a-ad8f-408ebd8c62cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99b488d577a4f9098769a745789f207, user_id=60effcfa14064f16aa7f98e286da3e42, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.574191 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd7542-ebce-4c4a-ad8f-408ebd8c62cd admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.581811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ae1c9-9822-411a-80e8-016d503d20df None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.586294 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1395/2787] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:15 2022] PUT /identity/v3/projects/b99b488d577a4f9098769a745789f207/users/60effcfa14064f16aa7f98e286da3e42/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:15.657837 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.658507 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.658862 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.659086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] PATH_INFO: `/v3/domains` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.659915 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80214) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:15.660230 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.661906 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.675338 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1393/2788] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:28:15.680113 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd7542-ebce-4c4a-ad8f-408ebd8c62cd admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.694095 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd7542-ebce-4c4a-ad8f-408ebd8c62cd admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.699318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81ba82-3511-41dc-b04e-9d1c4e22a0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.700012 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ba82-3511-41dc-b04e-9d1c4e22a0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.700456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ba82-3511-41dc-b04e-9d1c4e22a0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.700731 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ba82-3511-41dc-b04e-9d1c4e22a0f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.701472 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81ba82-3511-41dc-b04e-9d1c4e22a0f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.703712 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af81ba82-3511-41dc-b04e-9d1c4e22a0f0 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.734028 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1396/2789] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:15 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:15.738477 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.749084 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc92da-7e63-4c2d-bdeb-f817d247218c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.802589 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:15.824927 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3993aa-6d0b-4b3a-9d10-327527b52924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.825623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3993aa-6d0b-4b3a-9d10-327527b52924 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.825959 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3993aa-6d0b-4b3a-9d10-327527b52924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.826225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3993aa-6d0b-4b3a-9d10-327527b52924 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.826929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3993aa-6d0b-4b3a-9d10-327527b52924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.828896 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3993aa-6d0b-4b3a-9d10-327527b52924 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.857311 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1394/2790] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:15 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:15.858952 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81ba82-3511-41dc-b04e-9d1c4e22a0f0 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.873911 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af81ba82-3511-41dc-b04e-9d1c4e22a0f0 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.880237 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97789b0-760c-4acc-97a7-b45b7dd29540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.881067 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97789b0-760c-4acc-97a7-b45b7dd29540 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.881418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97789b0-760c-4acc-97a7-b45b7dd29540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.881799 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97789b0-760c-4acc-97a7-b45b7dd29540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.882602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97789b0-760c-4acc-97a7-b45b7dd29540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.884287 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97789b0-760c-4acc-97a7-b45b7dd29540 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.901236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1397/2791] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:15.903339 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3993aa-6d0b-4b3a-9d10-327527b52924 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.913867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3993aa-6d0b-4b3a-9d10-327527b52924 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.919444 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f83b6-4911-4133-b40e-96c829409c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.920123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f83b6-4911-4133-b40e-96c829409c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.920410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f83b6-4911-4133-b40e-96c829409c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.920659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f83b6-4911-4133-b40e-96c829409c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.921464 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f83b6-4911-4133-b40e-96c829409c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.923302 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f83b6-4911-4133-b40e-96c829409c4c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:15.937694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1395/2792] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:15.939543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97789b0-760c-4acc-97a7-b45b7dd29540 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:15.950881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97789b0-760c-4acc-97a7-b45b7dd29540 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:15.963006 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899b477-11af-4867-911e-ca1c4eda9d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:15.963971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b477-11af-4867-911e-ca1c4eda9d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:15.964939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b477-11af-4867-911e-ca1c4eda9d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:15.965179 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b477-11af-4867-911e-ca1c4eda9d25 admin admin] PATH_INFO: `/v3/projects/43a668013c4d4c24ab7a11eeaf301448/users/261a1ee643fe4704a7fddc2f3fb17940/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:15.969137 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:15.996300 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899b477-11af-4867-911e-ca1c4eda9d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43a668013c4d4c24ab7a11eeaf301448, user_id=261a1ee643fe4704a7fddc2f3fb17940, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:15.999196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899b477-11af-4867-911e-ca1c4eda9d25 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:16.015623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1398/2793] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:15 2022] PUT /identity/v3/projects/43a668013c4d4c24ab7a11eeaf301448/users/261a1ee643fe4704a7fddc2f3fb17940/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:16.026269 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6f20-eddb-4d0a-8ab8-137fec4f208b None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.027499 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6f20-eddb-4d0a-8ab8-137fec4f208b None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.028021 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac6f20-eddb-4d0a-8ab8-137fec4f208b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.062101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8aac6f20-eddb-4d0a-8ab8-137fec4f208b None None] MFA Rules not processed for user `261a1ee643fe4704a7fddc2f3fb17940`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:16.063403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8aac6f20-eddb-4d0a-8ab8-137fec4f208b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:16.141697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1396/2794] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:16.145481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899b477-11af-4867-911e-ca1c4eda9d25 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:16.244136 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899b477-11af-4867-911e-ca1c4eda9d25 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:16.250503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29dcfb5-216a-40ad-88cc-942bea8dcda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:16.251501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29dcfb5-216a-40ad-88cc-942bea8dcda8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.251501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29dcfb5-216a-40ad-88cc-942bea8dcda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.251831 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29dcfb5-216a-40ad-88cc-942bea8dcda8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.252436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29dcfb5-216a-40ad-88cc-942bea8dcda8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:16.254409 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29dcfb5-216a-40ad-88cc-942bea8dcda8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:16.292481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1399/2795] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:16 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:16.296895 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac6f20-eddb-4d0a-8ab8-137fec4f208b None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:16.313182 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac6f20-eddb-4d0a-8ab8-137fec4f208b None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:16.391774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596685a-ac87-4c40-a0da-c4fd83cea266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:16.392750 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596685a-ac87-4c40-a0da-c4fd83cea266 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.393188 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596685a-ac87-4c40-a0da-c4fd83cea266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.393524 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596685a-ac87-4c40-a0da-c4fd83cea266 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.394418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596685a-ac87-4c40-a0da-c4fd83cea266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:16.396676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596685a-ac87-4c40-a0da-c4fd83cea266 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:16.441252 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1397/2796] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:16 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:16.443600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29dcfb5-216a-40ad-88cc-942bea8dcda8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:16.473864 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29dcfb5-216a-40ad-88cc-942bea8dcda8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:16.482051 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89edbb-b61f-4bc2-b916-0f295ef8ee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:16.483366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89edbb-b61f-4bc2-b916-0f295ef8ee23 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.483717 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89edbb-b61f-4bc2-b916-0f295ef8ee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.484099 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89edbb-b61f-4bc2-b916-0f295ef8ee23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.485020 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89edbb-b61f-4bc2-b916-0f295ef8ee23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:16.487371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b89edbb-b61f-4bc2-b916-0f295ef8ee23 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:16.505437 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1400/2797] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:16.509541 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596685a-ac87-4c40-a0da-c4fd83cea266 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:16.521914 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596685a-ac87-4c40-a0da-c4fd83cea266 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:16.527008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a2f05-fbdd-4b0a-b12d-4bb28d6fc09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:16.527913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a2f05-fbdd-4b0a-b12d-4bb28d6fc09f admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.528205 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a2f05-fbdd-4b0a-b12d-4bb28d6fc09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.528502 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a2f05-fbdd-4b0a-b12d-4bb28d6fc09f admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.529259 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a2f05-fbdd-4b0a-b12d-4bb28d6fc09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:16.531456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a2f05-fbdd-4b0a-b12d-4bb28d6fc09f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:16.545703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1398/2798] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:16.548047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89edbb-b61f-4bc2-b916-0f295ef8ee23 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:16.559425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89edbb-b61f-4bc2-b916-0f295ef8ee23 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:16.565742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25147261-f8a4-49d5-8fad-91f4ac91e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:16.566695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25147261-f8a4-49d5-8fad-91f4ac91e1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.567631 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25147261-f8a4-49d5-8fad-91f4ac91e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.567917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25147261-f8a4-49d5-8fad-91f4ac91e1d5 admin admin] PATH_INFO: `/v3/projects/a9f658105d914802819ed9d66324a007/users/6d00ab02d47b47bab04c5e2c5d64abfa/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.598674 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25147261-f8a4-49d5-8fad-91f4ac91e1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f658105d914802819ed9d66324a007, user_id=6d00ab02d47b47bab04c5e2c5d64abfa, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:16.601649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25147261-f8a4-49d5-8fad-91f4ac91e1d5 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:16.621768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1401/2799] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:16 2022] PUT /identity/v3/projects/a9f658105d914802819ed9d66324a007/users/6d00ab02d47b47bab04c5e2c5d64abfa/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:16.632257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca8f83-7079-4ce4-b436-e1b31c3c8749 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.632860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca8f83-7079-4ce4-b436-e1b31c3c8749 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.632860 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca8f83-7079-4ce4-b436-e1b31c3c8749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.667648 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-cdca8f83-7079-4ce4-b436-e1b31c3c8749 None None] MFA Rules not processed for user `6d00ab02d47b47bab04c5e2c5d64abfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:16.668803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-cdca8f83-7079-4ce4-b436-e1b31c3c8749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:16.695968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:16.759890 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1399/2800] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:16.764879 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25147261-f8a4-49d5-8fad-91f4ac91e1d5 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:16.846104 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25147261-f8a4-49d5-8fad-91f4ac91e1d5 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:16.851432 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089eb06-25e8-4106-a55f-3a9ef7a760b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:16.852185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089eb06-25e8-4106-a55f-3a9ef7a760b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.852534 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089eb06-25e8-4106-a55f-3a9ef7a760b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.852872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089eb06-25e8-4106-a55f-3a9ef7a760b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.853649 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089eb06-25e8-4106-a55f-3a9ef7a760b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:16.855762 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089eb06-25e8-4106-a55f-3a9ef7a760b2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:16.889579 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1402/2801] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:16 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:16.891355 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca8f83-7079-4ce4-b436-e1b31c3c8749 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:16.907078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca8f83-7079-4ce4-b436-e1b31c3c8749 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:16.981048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8489ce4-438c-4210-ac58-9ea613d7afb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:16.981765 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8489ce4-438c-4210-ac58-9ea613d7afb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:16.982051 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8489ce4-438c-4210-ac58-9ea613d7afb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:16.982327 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8489ce4-438c-4210-ac58-9ea613d7afb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:16.984155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8489ce4-438c-4210-ac58-9ea613d7afb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:16.985758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8489ce4-438c-4210-ac58-9ea613d7afb0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:17.017344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1400/2802] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:16 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:17.018783 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089eb06-25e8-4106-a55f-3a9ef7a760b2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:17.031168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089eb06-25e8-4106-a55f-3a9ef7a760b2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:17.037880 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8024ac-1e56-4e90-8150-3d6b311d588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:17.038763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8024ac-1e56-4e90-8150-3d6b311d588b admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:17.039083 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8024ac-1e56-4e90-8150-3d6b311d588b admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:17.040209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8024ac-1e56-4e90-8150-3d6b311d588b admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:17.040209 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8024ac-1e56-4e90-8150-3d6b311d588b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:17.042115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8024ac-1e56-4e90-8150-3d6b311d588b admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:17.058420 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1403/2803] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:17.059856 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8489ce4-438c-4210-ac58-9ea613d7afb0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:17.073591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8489ce4-438c-4210-ac58-9ea613d7afb0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:17.080490 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39afd32-8973-4af0-ae0e-8a0a19853663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:17.081749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39afd32-8973-4af0-ae0e-8a0a19853663 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:17.081931 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39afd32-8973-4af0-ae0e-8a0a19853663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:17.082298 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39afd32-8973-4af0-ae0e-8a0a19853663 admin admin] PATH_INFO: `/v3/projects/a4cb74426eb749a39072d9b84d0a4fb4/users/3783fe97765d43c1b32b7b37506e8f97/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:17.101565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39afd32-8973-4af0-ae0e-8a0a19853663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cb74426eb749a39072d9b84d0a4fb4, user_id=3783fe97765d43c1b32b7b37506e8f97, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:17.104792 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39afd32-8973-4af0-ae0e-8a0a19853663 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:17.125362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1401/2804] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:17 2022] PUT /identity/v3/projects/a4cb74426eb749a39072d9b84d0a4fb4/users/3783fe97765d43c1b32b7b37506e8f97/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:17.126826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8024ac-1e56-4e90-8150-3d6b311d588b admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:17.139674 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8024ac-1e56-4e90-8150-3d6b311d588b admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:17.147536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:17.247742 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa2378-d4cb-43b2-9dab-f47fed277f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:17.248798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2378-d4cb-43b2-9dab-f47fed277f36 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:17.249275 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2378-d4cb-43b2-9dab-f47fed277f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:17.249739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa2378-d4cb-43b2-9dab-f47fed277f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:17.250958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa2378-d4cb-43b2-9dab-f47fed277f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:17.252939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa2378-d4cb-43b2-9dab-f47fed277f36 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:17.275888 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1404/2805] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:17.277017 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39afd32-8973-4af0-ae0e-8a0a19853663 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:17.289379 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39afd32-8973-4af0-ae0e-8a0a19853663 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:17.298195 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:17.299280 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:17.299703 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:17.300085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] PATH_INFO: `/v3/projects/a4cb74426eb749a39072d9b84d0a4fb4/users/3783fe97765d43c1b32b7b37506e8f97/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:17.302516 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cb74426eb749a39072d9b84d0a4fb4, user_id=3783fe97765d43c1b32b7b37506e8f97, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:17.305283 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:17.320963 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1402/2806] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:17 2022] PUT /identity/v3/projects/a4cb74426eb749a39072d9b84d0a4fb4/users/3783fe97765d43c1b32b7b37506e8f97/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:17.395582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe26a3-d5b7-4cac-bb8a-e5ec67358506 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:17.396702 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe26a3-d5b7-4cac-bb8a-e5ec67358506 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:17.396863 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe26a3-d5b7-4cac-bb8a-e5ec67358506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:17.436834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-5bfe26a3-d5b7-4cac-bb8a-e5ec67358506 None None] MFA Rules not processed for user `3783fe97765d43c1b32b7b37506e8f97`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:17.438671 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5bfe26a3-d5b7-4cac-bb8a-e5ec67358506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:17.554623 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1405/2807] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:17.562829 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:17.564869 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:17.650397 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d0f51-d5cb-42f4-ac32-6dfa47653bde admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:17.655191 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333473a3-b83f-4222-a728-4ad837a13ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:17.655935 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333473a3-b83f-4222-a728-4ad837a13ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:17.656346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333473a3-b83f-4222-a728-4ad837a13ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:17.656767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333473a3-b83f-4222-a728-4ad837a13ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:17.692211 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:17.738672 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333473a3-b83f-4222-a728-4ad837a13ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:17.740569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333473a3-b83f-4222-a728-4ad837a13ea9 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:17.752079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1403/2808] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:20.812056 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe26a3-d5b7-4cac-bb8a-e5ec67358506 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:20.813832 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-5bfe26a3-d5b7-4cac-bb8a-e5ec67358506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:20.828813 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe26a3-d5b7-4cac-bb8a-e5ec67358506 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:20.836797 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:20.837492 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:20.837829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:20.838115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:20.987299 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:20.989288 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:21.013295 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1406/2809] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:21.679088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6263b81-e3c2-4087-a784-9796d61006de None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:21.679714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6263b81-e3c2-4087-a784-9796d61006de None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:21.679714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6263b81-e3c2-4087-a784-9796d61006de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:21.719455 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-f6263b81-e3c2-4087-a784-9796d61006de None None] MFA Rules not processed for user `60effcfa14064f16aa7f98e286da3e42`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:21.720832 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f6263b81-e3c2-4087-a784-9796d61006de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:21.837128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1404/2810] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:21.854525 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:21.856240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:21.911850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:22.014046 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8248b-1b32-4a65-bca2-5a78a06674e1 service cinder] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:22.026740 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdddb0-acf2-4c75-ac9f-2202d60ff47f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:22.027848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdddb0-acf2-4c75-ac9f-2202d60ff47f service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:22.028141 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdddb0-acf2-4c75-ac9f-2202d60ff47f service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:22.028422 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdddb0-acf2-4c75-ac9f-2202d60ff47f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:22.030664 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9dcdddb0-acf2-4c75-ac9f-2202d60ff47f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:22.096121 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6263b81-e3c2-4087-a784-9796d61006de None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:22.110300 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6263b81-e3c2-4087-a784-9796d61006de None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:22.115248 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af15951-eb6c-44c2-96b8-5df7b5ba4770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:22.116114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15951-eb6c-44c2-96b8-5df7b5ba4770 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:22.116410 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15951-eb6c-44c2-96b8-5df7b5ba4770 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:22.116698 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af15951-eb6c-44c2-96b8-5df7b5ba4770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:22.120549 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdddb0-acf2-4c75-ac9f-2202d60ff47f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:22.122580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdddb0-acf2-4c75-ac9f-2202d60ff47f service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:22.128616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af15951-eb6c-44c2-96b8-5df7b5ba4770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:22.130214 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af15951-eb6c-44c2-96b8-5df7b5ba4770 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:22.138860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1407/2811] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:22.147411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1405/2812] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:28:23.258047 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45dbec-73a0-49b8-8579-e6014005408d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:23.258593 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45dbec-73a0-49b8-8579-e6014005408d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:23.258739 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45dbec-73a0-49b8-8579-e6014005408d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:23.279695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ca45dbec-73a0-49b8-8579-e6014005408d None None] MFA Rules not processed for user `0b41303f85c7455087c8d25658f1a548`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:23.280950 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ca45dbec-73a0-49b8-8579-e6014005408d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:23.289235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1408/2813] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:29.310088 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af15951-eb6c-44c2-96b8-5df7b5ba4770 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:29.374514 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af15951-eb6c-44c2-96b8-5df7b5ba4770 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:29.379273 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:29.380143 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:29.380419 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:29.380691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] PATH_INFO: `/v3/users/261a1ee643fe4704a7fddc2f3fb17940` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:29.381898 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=261a1ee643fe4704a7fddc2f3fb17940)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:29.383920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:29.435629 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:29.444161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '261a1ee643fe4704a7fddc2f3fb17940'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:29.456381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '261a1ee643fe4704a7fddc2f3fb17940'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:29.469522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '261a1ee643fe4704a7fddc2f3fb17940'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:29.470321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '261a1ee643fe4704a7fddc2f3fb17940'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:29.470952 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '261a1ee643fe4704a7fddc2f3fb17940'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:29.484578 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1406/2814] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:29 2022] DELETE /identity/v3/users/261a1ee643fe4704a7fddc2f3fb17940 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:29.489397 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692efd-1f04-40ac-8c46-9d3ecb6b3013 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:29.489905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692efd-1f04-40ac-8c46-9d3ecb6b3013 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:29.490005 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692efd-1f04-40ac-8c46-9d3ecb6b3013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:29.508932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-40692efd-1f04-40ac-8c46-9d3ecb6b3013 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:29.510202 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-40692efd-1f04-40ac-8c46-9d3ecb6b3013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:29.570378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1409/2815] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:29.582835 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:29.584761 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:29.658759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe983401-7e15-4aeb-8f87-5cc1ea2fcdf0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:29.662607 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038850a-f60a-43f1-89a1-00722c8ee6f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:29.663267 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038850a-f60a-43f1-89a1-00722c8ee6f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:29.663553 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038850a-f60a-43f1-89a1-00722c8ee6f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:29.663834 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038850a-f60a-43f1-89a1-00722c8ee6f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:29.675942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038850a-f60a-43f1-89a1-00722c8ee6f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:29.677573 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038850a-f60a-43f1-89a1-00722c8ee6f0 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:29.688065 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1407/2816] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:30.008244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40692efd-1f04-40ac-8c46-9d3ecb6b3013 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:30.010464 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-40692efd-1f04-40ac-8c46-9d3ecb6b3013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:30.024523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40692efd-1f04-40ac-8c46-9d3ecb6b3013 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:30.030344 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:30.031497 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:30.031764 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:30.032039 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] PATH_INFO: `/v3/projects/43a668013c4d4c24ab7a11eeaf301448` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:30.033364 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43a668013c4d4c24ab7a11eeaf301448)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:30.035395 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:30.075043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:30.090079 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43a668013c4d4c24ab7a11eeaf301448', 'request_id': 'req-c0bfad83-826e-4dab-aae5-4cca801155f7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.101874 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43a668013c4d4c24ab7a11eeaf301448', 'request_id': 'req-c0bfad83-826e-4dab-aae5-4cca801155f7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.110385 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43a668013c4d4c24ab7a11eeaf301448', 'request_id': 'req-c0bfad83-826e-4dab-aae5-4cca801155f7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.123694 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] The token cache is being invalidate because project 43a668013c4d4c24ab7a11eeaf301448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:30.124286 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.125248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.127814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1410/2817] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:30 2022] DELETE /identity/v3/projects/43a668013c4d4c24ab7a11eeaf301448 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:30.132246 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038850a-f60a-43f1-89a1-00722c8ee6f0 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:30.157536 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038850a-f60a-43f1-89a1-00722c8ee6f0 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:30.227189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:30.228094 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:30.228391 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:30.228740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] PATH_INFO: `/v3/users/6d00ab02d47b47bab04c5e2c5d64abfa` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:30.230401 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d00ab02d47b47bab04c5e2c5d64abfa)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:30.232537 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:30.291853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d00ab02d47b47bab04c5e2c5d64abfa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.301918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d00ab02d47b47bab04c5e2c5d64abfa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.310631 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:30.324970 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d00ab02d47b47bab04c5e2c5d64abfa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.326150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d00ab02d47b47bab04c5e2c5d64abfa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.327016 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d00ab02d47b47bab04c5e2c5d64abfa'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.341426 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1408/2818] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:30 2022] DELETE /identity/v3/users/6d00ab02d47b47bab04c5e2c5d64abfa => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:30.652842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:30.654532 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:30.668183 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfad83-826e-4dab-aae5-4cca801155f7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:30.740871 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:30.741887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:30.741985 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:30.742251 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] PATH_INFO: `/v3/projects/a9f658105d914802819ed9d66324a007` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:30.743533 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f658105d914802819ed9d66324a007)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:30.745650 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:30.829638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f658105d914802819ed9d66324a007', 'request_id': 'req-096c3842-004c-4d2b-9a64-bf0557ffcf4e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.842110 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f658105d914802819ed9d66324a007', 'request_id': 'req-096c3842-004c-4d2b-9a64-bf0557ffcf4e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.851236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f658105d914802819ed9d66324a007', 'request_id': 'req-096c3842-004c-4d2b-9a64-bf0557ffcf4e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.865607 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] The token cache is being invalidate because project a9f658105d914802819ed9d66324a007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:30.865859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.866638 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:30.868479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1411/2819] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:30 2022] DELETE /identity/v3/projects/a9f658105d914802819ed9d66324a007 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:30.872987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:30.875060 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:30.894572 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4751a-fedf-465f-81a6-09320221beb2 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:30.969042 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:30.970035 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:30.970250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:30.970525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] PATH_INFO: `/v3/users/3783fe97765d43c1b32b7b37506e8f97` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:30.972037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:30.972189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3783fe97765d43c1b32b7b37506e8f97)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:30.973048 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:30.973853 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:30.999726 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c3842-004c-4d2b-9a64-bf0557ffcf4e admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:31.008619 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:31.009953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.010373 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.010810 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] PATH_INFO: `/v3/users/c50bba5970f54de49a433fe158b895fa` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.013435 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c50bba5970f54de49a433fe158b895fa)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:31.016792 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:31.040678 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3783fe97765d43c1b32b7b37506e8f97'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.058740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3783fe97765d43c1b32b7b37506e8f97'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.079079 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3783fe97765d43c1b32b7b37506e8f97'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.081241 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3783fe97765d43c1b32b7b37506e8f97'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.082533 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-03a1a19e-d546-4ca9-8c5e-19b10fe35e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3783fe97765d43c1b32b7b37506e8f97'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.102926 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1409/2820] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:30 2022] DELETE /identity/v3/users/3783fe97765d43c1b32b7b37506e8f97 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:31.118673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:31.123282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c50bba5970f54de49a433fe158b895fa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.135108 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c50bba5970f54de49a433fe158b895fa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.152417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c50bba5970f54de49a433fe158b895fa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.152417 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c50bba5970f54de49a433fe158b895fa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.153171 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c50bba5970f54de49a433fe158b895fa'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.178428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1412/2821] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:30 2022] DELETE /identity/v3/users/c50bba5970f54de49a433fe158b895fa => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:31.184155 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe087f53-ebc9-4b90-8578-9b8edbe4b2ac None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.184933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe087f53-ebc9-4b90-8578-9b8edbe4b2ac None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.185154 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe087f53-ebc9-4b90-8578-9b8edbe4b2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.212665 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fe087f53-ebc9-4b90-8578-9b8edbe4b2ac None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:31.214847 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fe087f53-ebc9-4b90-8578-9b8edbe4b2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:31.303167 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:31.326651 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1410/2822] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:31.342869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:31.346390 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:31.441684 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385d1ac-ca9d-45d5-8297-7277ca61eb3c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:31.447371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2bd0f-4365-4b4d-8224-771149cd2a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:31.448115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2bd0f-4365-4b4d-8224-771149cd2a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.448401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2bd0f-4365-4b4d-8224-771149cd2a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.449895 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2bd0f-4365-4b4d-8224-771149cd2a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.461710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2bd0f-4365-4b4d-8224-771149cd2a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:31.464169 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2bd0f-4365-4b4d-8224-771149cd2a25 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:31.482350 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1413/2823] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:31.516015 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe087f53-ebc9-4b90-8578-9b8edbe4b2ac None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:31.518003 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fe087f53-ebc9-4b90-8578-9b8edbe4b2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:31.537797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe087f53-ebc9-4b90-8578-9b8edbe4b2ac None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:31.545131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:31.546276 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.546623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.546958 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] PATH_INFO: `/v3/projects/a4cb74426eb749a39072d9b84d0a4fb4` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.548324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4cb74426eb749a39072d9b84d0a4fb4)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:31.551402 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:31.613096 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4cb74426eb749a39072d9b84d0a4fb4', 'request_id': 'req-9843082a-dd7f-4092-9465-4425419cadd2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.621569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4cb74426eb749a39072d9b84d0a4fb4', 'request_id': 'req-9843082a-dd7f-4092-9465-4425419cadd2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.634403 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4cb74426eb749a39072d9b84d0a4fb4', 'request_id': 'req-9843082a-dd7f-4092-9465-4425419cadd2'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.656288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] The token cache is being invalidate because project a4cb74426eb749a39072d9b84d0a4fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:31.656646 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.657615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-9843082a-dd7f-4092-9465-4425419cadd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:31.662460 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1411/2824] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:31 2022] DELETE /identity/v3/projects/a4cb74426eb749a39072d9b84d0a4fb4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:31.670590 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655506f6-edb1-42ee-87d4-3aae52fb6c89 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.671134 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655506f6-edb1-42ee-87d4-3aae52fb6c89 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.671523 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655506f6-edb1-42ee-87d4-3aae52fb6c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.692860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-655506f6-edb1-42ee-87d4-3aae52fb6c89 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:31.694158 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-655506f6-edb1-42ee-87d4-3aae52fb6c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:31.787318 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1414/2825] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:31.790809 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1edca2-002b-4c2b-96d4-9db095d858b5 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.791098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1edca2-002b-4c2b-96d4-9db095d858b5 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.791378 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1edca2-002b-4c2b-96d4-9db095d858b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.815735 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-8b1edca2-002b-4c2b-96d4-9db095d858b5 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:31.817734 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8b1edca2-002b-4c2b-96d4-9db095d858b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:31.830949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1412/2826] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:31.835775 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655506f6-edb1-42ee-87d4-3aae52fb6c89 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:31.853550 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655506f6-edb1-42ee-87d4-3aae52fb6c89 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:31.867154 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:31.867957 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.868767 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.868945 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.870535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:31.871054 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:31.873970 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:31.894255 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1415/2827] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:28:31.901995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1edca2-002b-4c2b-96d4-9db095d858b5 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:31.919363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1edca2-002b-4c2b-96d4-9db095d858b5 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:31.930036 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d89c5e-260e-4a5c-a985-55fc2d371b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:31.930984 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d89c5e-260e-4a5c-a985-55fc2d371b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.931345 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d89c5e-260e-4a5c-a985-55fc2d371b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.931941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d89c5e-260e-4a5c-a985-55fc2d371b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.932755 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d89c5e-260e-4a5c-a985-55fc2d371b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:31.936180 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d89c5e-260e-4a5c-a985-55fc2d371b09 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:31.947120 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:31.950011 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:31.982227 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e5c1e-eb72-4244-afbf-7ec08a6c8e60 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:31.991021 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:31.991816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1413/2828] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:31 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:31.992678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:31.993266 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:31.993914 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] PATH_INFO: `/v3/projects/6e81930c6a9e4cd98f48cfe672c509af` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:31.996549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d89c5e-260e-4a5c-a985-55fc2d371b09 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:31.997075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e81930c6a9e4cd98f48cfe672c509af)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.001057 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.080567 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:32.082768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e81930c6a9e4cd98f48cfe672c509af', 'request_id': 'req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:32.083219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d89c5e-260e-4a5c-a985-55fc2d371b09 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.094602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e81930c6a9e4cd98f48cfe672c509af', 'request_id': 'req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:32.109347 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e81930c6a9e4cd98f48cfe672c509af', 'request_id': 'req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:32.131559 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] The token cache is being invalidate because project 6e81930c6a9e4cd98f48cfe672c509af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:32.131885 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:32.132601 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-8dc2e974-0c4f-4d37-bf18-7969d52cc21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:32.134279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1416/2829] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:31 2022] DELETE /identity/v3/projects/6e81930c6a9e4cd98f48cfe672c509af => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:32.139398 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18444-6bd4-494c-8bdb-ea7b1148ff27 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.139683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18444-6bd4-494c-8bdb-ea7b1148ff27 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.139959 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd18444-6bd4-494c-8bdb-ea7b1148ff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.160092 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-ccd18444-6bd4-494c-8bdb-ea7b1148ff27 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:32.160801 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.161323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-ccd18444-6bd4-494c-8bdb-ea7b1148ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:32.161485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.161810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.162803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.162803 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.164759 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.170427 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1417/2830] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:32.172190 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:32.179933 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16308e-75fa-4d4d-ad41-16de790fc232 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.180331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16308e-75fa-4d4d-ad41-16de790fc232 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.180519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16308e-75fa-4d4d-ad41-16de790fc232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.201987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1414/2831] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:31 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:32.207957 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.210098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:32.212829 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-1d16308e-75fa-4d4d-ad41-16de790fc232 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:32.214665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1d16308e-75fa-4d4d-ad41-16de790fc232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:32.224480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1418/2832] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:32.225172 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c72ec-6d01-4711-ae16-a7d1980de63f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.228998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16308e-75fa-4d4d-ad41-16de790fc232 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.230697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35e387-ef96-4ddb-80eb-3f77530e4b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.231484 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35e387-ef96-4ddb-80eb-3f77530e4b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.231839 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35e387-ef96-4ddb-80eb-3f77530e4b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.232181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d35e387-ef96-4ddb-80eb-3f77530e4b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.233389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35e387-ef96-4ddb-80eb-3f77530e4b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.235392 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d35e387-ef96-4ddb-80eb-3f77530e4b10 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.240636 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16308e-75fa-4d4d-ad41-16de790fc232 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.247218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1415/2833] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:32.251418 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.251827 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.252204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.252656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.253152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35e387-ef96-4ddb-80eb-3f77530e4b10 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.253441 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:32.253819 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.255782 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.265153 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d35e387-ef96-4ddb-80eb-3f77530e4b10 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.270686 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1419/2834] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 18 14:28:32.271234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d424ae-3f8d-45d4-8300-7f8b2ebba1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.272102 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d424ae-3f8d-45d4-8300-7f8b2ebba1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.272380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d424ae-3f8d-45d4-8300-7f8b2ebba1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.272656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d424ae-3f8d-45d4-8300-7f8b2ebba1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.273395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d424ae-3f8d-45d4-8300-7f8b2ebba1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.274816 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.275261 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d424ae-3f8d-45d4-8300-7f8b2ebba1c4 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.286910 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5cb194-fef7-4d40-be1e-c5491f469265 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.294463 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47540aa-e896-450c-a223-b8b6551bdd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.295008 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1416/2835] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:32.295734 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47540aa-e896-450c-a223-b8b6551bdd12 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.296198 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47540aa-e896-450c-a223-b8b6551bdd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.296483 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47540aa-e896-450c-a223-b8b6551bdd12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.297193 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47540aa-e896-450c-a223-b8b6551bdd12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.299078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d424ae-3f8d-45d4-8300-7f8b2ebba1c4 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.299682 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47540aa-e896-450c-a223-b8b6551bdd12 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.317254 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d424ae-3f8d-45d4-8300-7f8b2ebba1c4 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.322616 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f194c-fb78-4386-85a5-6954f92e3994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.323859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f194c-fb78-4386-85a5-6954f92e3994 admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.324192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f194c-fb78-4386-85a5-6954f92e3994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.324394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f194c-fb78-4386-85a5-6954f92e3994 admin admin] PATH_INFO: `/v3/projects/7a000079ba214fdf934764899139f62d/users/7093c86da40243a5a8e50ad0f4cd33bb/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.335582 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1420/2836] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:32 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:32.340321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47540aa-e896-450c-a223-b8b6551bdd12 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.342556 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f194c-fb78-4386-85a5-6954f92e3994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a000079ba214fdf934764899139f62d, user_id=7093c86da40243a5a8e50ad0f4cd33bb, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.345422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f194c-fb78-4386-85a5-6954f92e3994 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.364181 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1417/2837] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:32 2022] PUT /identity/v3/projects/7a000079ba214fdf934764899139f62d/users/7093c86da40243a5a8e50ad0f4cd33bb/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:32.370394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c1f7-3058-472d-8a00-010d842219c7 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.370608 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c1f7-3058-472d-8a00-010d842219c7 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.370899 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51c1f7-3058-472d-8a00-010d842219c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.396029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-fa51c1f7-3058-472d-8a00-010d842219c7 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:32.397168 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fa51c1f7-3058-472d-8a00-010d842219c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:32.487744 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1418/2838] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:32.499504 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c1f7-3058-472d-8a00-010d842219c7 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.501009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-fa51c1f7-3058-472d-8a00-010d842219c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:32.547389 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47540aa-e896-450c-a223-b8b6551bdd12 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.553212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f64f4-ae7b-42fe-b15e-481935fe02aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.553947 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f64f4-ae7b-42fe-b15e-481935fe02aa admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.554232 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f64f4-ae7b-42fe-b15e-481935fe02aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.554506 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1f64f4-ae7b-42fe-b15e-481935fe02aa admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.555203 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f64f4-ae7b-42fe-b15e-481935fe02aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.557216 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1f64f4-ae7b-42fe-b15e-481935fe02aa admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.568221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51c1f7-3058-472d-8a00-010d842219c7 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.573812 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf86e93-a65a-4e52-aa0e-c12ee98f049d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.574493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf86e93-a65a-4e52-aa0e-c12ee98f049d service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.574817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf86e93-a65a-4e52-aa0e-c12ee98f049d service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.575123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf86e93-a65a-4e52-aa0e-c12ee98f049d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.580428 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1421/2839] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:32 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 18 14:28:32.584661 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f64f4-ae7b-42fe-b15e-481935fe02aa admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.587988 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf86e93-a65a-4e52-aa0e-c12ee98f049d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.589836 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf86e93-a65a-4e52-aa0e-c12ee98f049d service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.596022 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1f64f4-ae7b-42fe-b15e-481935fe02aa admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.600860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea40aaa-6929-4428-bed8-fca191d9c3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.601038 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1419/2840] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:28:32.601555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea40aaa-6929-4428-bed8-fca191d9c3ab admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.601968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea40aaa-6929-4428-bed8-fca191d9c3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.602204 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea40aaa-6929-4428-bed8-fca191d9c3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.603085 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea40aaa-6929-4428-bed8-fca191d9c3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.604818 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea40aaa-6929-4428-bed8-fca191d9c3ab admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.619514 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1422/2841] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:32.621523 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf86e93-a65a-4e52-aa0e-c12ee98f049d service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.632874 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf86e93-a65a-4e52-aa0e-c12ee98f049d service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.639044 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78365a-e340-4372-a2c1-52996402a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.639683 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78365a-e340-4372-a2c1-52996402a0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.640039 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78365a-e340-4372-a2c1-52996402a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.640411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78365a-e340-4372-a2c1-52996402a0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.641221 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78365a-e340-4372-a2c1-52996402a0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.643144 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78365a-e340-4372-a2c1-52996402a0fb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.655664 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1420/2842] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:32.661805 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea40aaa-6929-4428-bed8-fca191d9c3ab admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.672937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea40aaa-6929-4428-bed8-fca191d9c3ab admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:32.680408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:32.681302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.681586 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.681922 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] PATH_INFO: `/v3/projects/92ae139e7dd9412594a48a975cfe5925/users/ea1952f349eb450199fe6a24fafb3c6d/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.701007 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ae139e7dd9412594a48a975cfe5925, user_id=ea1952f349eb450199fe6a24fafb3c6d, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:32.703936 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:32.725284 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1423/2843] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:32 2022] PUT /identity/v3/projects/92ae139e7dd9412594a48a975cfe5925/users/ea1952f349eb450199fe6a24fafb3c6d/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:32.737219 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b025e0-2af3-414d-9ec3-d2c7c3219c16 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:32.737546 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b025e0-2af3-414d-9ec3-d2c7c3219c16 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:32.737883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b025e0-2af3-414d-9ec3-d2c7c3219c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:32.786078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-b7b025e0-2af3-414d-9ec3-d2c7c3219c16 None None] MFA Rules not processed for user `ea1952f349eb450199fe6a24fafb3c6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:32.788324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-b7b025e0-2af3-414d-9ec3-d2c7c3219c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:32.823660 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:32.892326 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1421/2844] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:32.913806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:32.916951 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:32.987595 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:33.001589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7cfbf-9989-4c24-8eb1-6fdf6d373a2d admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:33.006868 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56bb9e-3d19-4a74-a0f3-2be3e593e657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:33.007756 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56bb9e-3d19-4a74-a0f3-2be3e593e657 service cinder] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:33.008144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56bb9e-3d19-4a74-a0f3-2be3e593e657 service cinder] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:33.008939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56bb9e-3d19-4a74-a0f3-2be3e593e657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:33.024958 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56bb9e-3d19-4a74-a0f3-2be3e593e657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:33.027997 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56bb9e-3d19-4a74-a0f3-2be3e593e657 service cinder] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:33.041168 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1424/2845] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:33.070525 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b025e0-2af3-414d-9ec3-d2c7c3219c16 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:33.153344 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b025e0-2af3-414d-9ec3-d2c7c3219c16 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:33.158636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:33.159570 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:33.159823 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:33.160120 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] PATH_INFO: `/v3/users/ea1952f349eb450199fe6a24fafb3c6d` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:33.161404 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1952f349eb450199fe6a24fafb3c6d)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:33.163395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:33.221145 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1952f349eb450199fe6a24fafb3c6d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:33.230987 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1952f349eb450199fe6a24fafb3c6d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:33.250329 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1952f349eb450199fe6a24fafb3c6d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:33.251579 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1952f349eb450199fe6a24fafb3c6d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:33.252255 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1952f349eb450199fe6a24fafb3c6d'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:33.267496 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1422/2846] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:33 2022] DELETE /identity/v3/users/ea1952f349eb450199fe6a24fafb3c6d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:33.272217 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7d596-1f35-42a3-a566-877152c35ea0 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:33.277315 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7d596-1f35-42a3-a566-877152c35ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:33.277675 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7d596-1f35-42a3-a566-877152c35ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:33.298654 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-0de7d596-1f35-42a3-a566-877152c35ea0 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:33.300219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0de7d596-1f35-42a3-a566-877152c35ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:33.407865 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1425/2847] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:33.418151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:33.420192 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:33.472604 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:33.491082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b5f1d-3279-43db-ab03-830e8221622d admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:33.494964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c15763-e157-4b3d-b6b7-e63ed9fb84f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:33.495713 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c15763-e157-4b3d-b6b7-e63ed9fb84f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:33.496070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c15763-e157-4b3d-b6b7-e63ed9fb84f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:33.496420 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c15763-e157-4b3d-b6b7-e63ed9fb84f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:33.507934 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c15763-e157-4b3d-b6b7-e63ed9fb84f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:33.509480 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c15763-e157-4b3d-b6b7-e63ed9fb84f1 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:33.525851 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1423/2848] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 18 14:28:33.896964 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7d596-1f35-42a3-a566-877152c35ea0 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:33.901321 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-0de7d596-1f35-42a3-a566-877152c35ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:33.935581 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7d596-1f35-42a3-a566-877152c35ea0 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:33.942419 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:33.943455 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:33.943869 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:33.944221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] PATH_INFO: `/v3/projects/92ae139e7dd9412594a48a975cfe5925` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:33.945872 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ae139e7dd9412594a48a975cfe5925)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:33.947988 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:34.013681 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ae139e7dd9412594a48a975cfe5925', 'request_id': 'req-f05b618e-df57-4ff7-95fd-e75a67e5383c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:34.027555 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ae139e7dd9412594a48a975cfe5925', 'request_id': 'req-f05b618e-df57-4ff7-95fd-e75a67e5383c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:34.037776 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ae139e7dd9412594a48a975cfe5925', 'request_id': 'req-f05b618e-df57-4ff7-95fd-e75a67e5383c'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:34.066127 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] The token cache is being invalidate because project 92ae139e7dd9412594a48a975cfe5925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:34.066426 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:34.067157 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:34.069704 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1426/2849] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:33 2022] DELETE /identity/v3/projects/92ae139e7dd9412594a48a975cfe5925 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:37.207991 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a68fc-6a4e-407e-b366-bd2ac52c0a70 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:37.208580 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a68fc-6a4e-407e-b366-bd2ac52c0a70 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:37.208808 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a68fc-6a4e-407e-b366-bd2ac52c0a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:37.245189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-e64a68fc-6a4e-407e-b366-bd2ac52c0a70 None None] MFA Rules not processed for user `7093c86da40243a5a8e50ad0f4cd33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:37.246656 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-e64a68fc-6a4e-407e-b366-bd2ac52c0a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:37.339776 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1424/2850] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:37.344777 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:37.346265 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:37.428809 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b618e-df57-4ff7-95fd-e75a67e5383c admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:37.433980 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dc2a6-ad73-465c-8fe5-1bcca2fc2da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:37.434677 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dc2a6-ad73-465c-8fe5-1bcca2fc2da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:37.434955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dc2a6-ad73-465c-8fe5-1bcca2fc2da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:37.435225 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dc2a6-ad73-465c-8fe5-1bcca2fc2da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:37.435934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dc2a6-ad73-465c-8fe5-1bcca2fc2da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:37.437917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dc2a6-ad73-465c-8fe5-1bcca2fc2da8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:37.453185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:37.473569 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1427/2851] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:37 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:37.479633 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a68fc-6a4e-407e-b366-bd2ac52c0a70 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:37.496206 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a68fc-6a4e-407e-b366-bd2ac52c0a70 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:37.567547 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1879-ad99-4593-b868-9a8fb240829c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:37.568278 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1879-ad99-4593-b868-9a8fb240829c admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:37.568565 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1879-ad99-4593-b868-9a8fb240829c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:37.568932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e1879-ad99-4593-b868-9a8fb240829c admin admin] PATH_INFO: `/v3/users` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:37.569636 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e1879-ad99-4593-b868-9a8fb240829c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:37.573529 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e1879-ad99-4593-b868-9a8fb240829c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:37.594288 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1425/2852] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:37 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:37.599505 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dc2a6-ad73-465c-8fe5-1bcca2fc2da8 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:37.620835 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dc2a6-ad73-465c-8fe5-1bcca2fc2da8 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:37.631392 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76db99-0db3-488d-a87b-0a599840c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:37.632107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76db99-0db3-488d-a87b-0a599840c473 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:37.632379 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76db99-0db3-488d-a87b-0a599840c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:37.632656 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76db99-0db3-488d-a87b-0a599840c473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:37.633401 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76db99-0db3-488d-a87b-0a599840c473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:37.635886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76db99-0db3-488d-a87b-0a599840c473 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:37.649971 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1428/2853] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:37.654576 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1879-ad99-4593-b868-9a8fb240829c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:37.666196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e1879-ad99-4593-b868-9a8fb240829c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:37.671476 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722056c0-e70f-4261-84a4-705392416bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:37.672390 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722056c0-e70f-4261-84a4-705392416bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:37.672671 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722056c0-e70f-4261-84a4-705392416bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:37.672947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722056c0-e70f-4261-84a4-705392416bdb admin admin] PATH_INFO: `/v3/projects/766435eed1ca42d697b0c97032bc40d4/users/f24bc48b705f4f4dbb3ad78e29dead11/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:37.694712 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722056c0-e70f-4261-84a4-705392416bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=766435eed1ca42d697b0c97032bc40d4, user_id=f24bc48b705f4f4dbb3ad78e29dead11, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:37.700347 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722056c0-e70f-4261-84a4-705392416bdb admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:37.722194 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1426/2854] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:37 2022] PUT /identity/v3/projects/766435eed1ca42d697b0c97032bc40d4/users/f24bc48b705f4f4dbb3ad78e29dead11/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:37.737920 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76db99-0db3-488d-a87b-0a599840c473 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:37.752850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76db99-0db3-488d-a87b-0a599840c473 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:37.828929 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4681f7-1061-4ddd-9e0f-9594876c18f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:37.829524 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4681f7-1061-4ddd-9e0f-9594876c18f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:37.829850 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4681f7-1061-4ddd-9e0f-9594876c18f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:37.830130 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4681f7-1061-4ddd-9e0f-9594876c18f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:37.830875 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4681f7-1061-4ddd-9e0f-9594876c18f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:37.832519 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4681f7-1061-4ddd-9e0f-9594876c18f3 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:37.844651 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1429/2855] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:37.849697 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722056c0-e70f-4261-84a4-705392416bdb admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:37.862998 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722056c0-e70f-4261-84a4-705392416bdb admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:37.868228 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:37.869196 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:37.869714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:37.870062 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] PATH_INFO: `/v3/projects/766435eed1ca42d697b0c97032bc40d4/users/f24bc48b705f4f4dbb3ad78e29dead11/roles/5c672a1b2979444383c587a215adeb35` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:37.872456 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=766435eed1ca42d697b0c97032bc40d4, user_id=f24bc48b705f4f4dbb3ad78e29dead11, role_id=5c672a1b2979444383c587a215adeb35)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:37.875360 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:37.892159 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1427/2856] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:37 2022] PUT /identity/v3/projects/766435eed1ca42d697b0c97032bc40d4/users/f24bc48b705f4f4dbb3ad78e29dead11/roles/5c672a1b2979444383c587a215adeb35 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:44.059858 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c32522-b08f-4e10-9e8c-0df6cff2856d None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:44.060480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c32522-b08f-4e10-9e8c-0df6cff2856d None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:44.060480 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c32522-b08f-4e10-9e8c-0df6cff2856d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:44.096962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-87c32522-b08f-4e10-9e8c-0df6cff2856d None None] MFA Rules not processed for user `f24bc48b705f4f4dbb3ad78e29dead11`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:44.098357 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-87c32522-b08f-4e10-9e8c-0df6cff2856d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:44.197887 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1430/2857] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:44.215029 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:44.216859 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:44.263277 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:44.330714 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712281b0-0729-4648-99c4-209de9d0fe6c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:44.336785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de263efb-e73d-44d6-a3b9-6ffd26c3f957 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:44.337425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de263efb-e73d-44d6-a3b9-6ffd26c3f957 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:44.337826 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de263efb-e73d-44d6-a3b9-6ffd26c3f957 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:44.338114 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de263efb-e73d-44d6-a3b9-6ffd26c3f957 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:44.432953 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de263efb-e73d-44d6-a3b9-6ffd26c3f957 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:44.436118 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de263efb-e73d-44d6-a3b9-6ffd26c3f957 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:44.460199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1428/2858] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:54.189345 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:INSERT=1,keystone:SELECT=70 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:54.452522 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=43 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:55.442909 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c32522-b08f-4e10-9e8c-0df6cff2856d None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:55.445053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-87c32522-b08f-4e10-9e8c-0df6cff2856d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:55.520616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c32522-b08f-4e10-9e8c-0df6cff2856d None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:55.525825 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:55.527593 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:55.527881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:55.528160 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] PATH_INFO: `/v3/users/288bcc4d12374626b1b9802a811a61f4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:55.529693 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=288bcc4d12374626b1b9802a811a61f4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:55.531724 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:55.599928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '288bcc4d12374626b1b9802a811a61f4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:55.609894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '288bcc4d12374626b1b9802a811a61f4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:55.626246 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '288bcc4d12374626b1b9802a811a61f4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:55.627075 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '288bcc4d12374626b1b9802a811a61f4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:55.627778 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '288bcc4d12374626b1b9802a811a61f4'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:55.649377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1431/2859] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:28:55 2022] DELETE /identity/v3/users/288bcc4d12374626b1b9802a811a61f4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:55.654115 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055268-8ed8-42f6-bc2b-b61a5de33a26 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:55.654677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055268-8ed8-42f6-bc2b-b61a5de33a26 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:55.654677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a055268-8ed8-42f6-bc2b-b61a5de33a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:55.672913 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-6a055268-8ed8-42f6-bc2b-b61a5de33a26 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:55.674073 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6a055268-8ed8-42f6-bc2b-b61a5de33a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:55.749563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1429/2860] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:55.764403 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:55.766181 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:55.830346 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b2163-750d-4dcb-a6ba-020070ebf8e7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:55.834812 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e6d01-49b3-4b51-8ad3-30a46b07a509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:55.835486 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e6d01-49b3-4b51-8ad3-30a46b07a509 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:55.835768 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e6d01-49b3-4b51-8ad3-30a46b07a509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:55.836103 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e6d01-49b3-4b51-8ad3-30a46b07a509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:55.847167 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e6d01-49b3-4b51-8ad3-30a46b07a509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:55.848648 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9e6d01-49b3-4b51-8ad3-30a46b07a509 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:55.860377 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1432/2861] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:28:56.185098 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a055268-8ed8-42f6-bc2b-b61a5de33a26 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.187095 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6a055268-8ed8-42f6-bc2b-b61a5de33a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:56.217743 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a055268-8ed8-42f6-bc2b-b61a5de33a26 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:56.224686 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:56.225865 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.226210 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.226551 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] PATH_INFO: `/v3/projects/b7d42244ad82492aadf1f51d7ad74f33` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.227920 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d42244ad82492aadf1f51d7ad74f33)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:56.230549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:56.295271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d42244ad82492aadf1f51d7ad74f33', 'request_id': 'req-b48dd416-65e6-444c-93e9-ce5103e2f959'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:56.302947 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d42244ad82492aadf1f51d7ad74f33', 'request_id': 'req-b48dd416-65e6-444c-93e9-ce5103e2f959'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:56.312758 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d42244ad82492aadf1f51d7ad74f33', 'request_id': 'req-b48dd416-65e6-444c-93e9-ce5103e2f959'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:56.327068 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] The token cache is being invalidate because project b7d42244ad82492aadf1f51d7ad74f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:28:56.327503 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:56.328324 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-b48dd416-65e6-444c-93e9-ce5103e2f959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:28:56.330116 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1430/2862] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:28:56 2022] DELETE /identity/v3/projects/b7d42244ad82492aadf1f51d7ad74f33 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:28:56.335218 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eabd8-ed00-42d5-995f-f4c58825e0aa None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.335512 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eabd8-ed00-42d5-995f-f4c58825e0aa None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.335788 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eabd8-ed00-42d5-995f-f4c58825e0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.355186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-269eabd8-ed00-42d5-995f-f4c58825e0aa None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:56.356405 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-269eabd8-ed00-42d5-995f-f4c58825e0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:56.380185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:56.437282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1433/2863] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:56.445310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa38fc-2e4d-4073-9315-a2e783db0f66 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.445569 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa38fc-2e4d-4073-9315-a2e783db0f66 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.445973 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa38fc-2e4d-4073-9315-a2e783db0f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.468018 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-62aa38fc-2e4d-4073-9315-a2e783db0f66 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:56.469472 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-62aa38fc-2e4d-4073-9315-a2e783db0f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:56.481193 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1431/2864] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 18 14:28:56.486087 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eabd8-ed00-42d5-995f-f4c58825e0aa None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.497123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eabd8-ed00-42d5-995f-f4c58825e0aa None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:56.502143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:56.502823 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.503109 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.503387 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.504124 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80213) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 18 14:28:56.504434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:56.506236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:56.522370 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1434/2865] 10.222.0.71 () {58 vars in 1152 bytes} [Mon Jul 18 14:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 18 14:28:56.526791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa38fc-2e4d-4073-9315-a2e783db0f66 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.536615 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa38fc-2e4d-4073-9315-a2e783db0f66 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:56.542199 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d6d5e-598f-4cf2-9593-f5617612aad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:56.542870 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d6d5e-598f-4cf2-9593-f5617612aad8 admin admin] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.543158 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d6d5e-598f-4cf2-9593-f5617612aad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.543438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d6d5e-598f-4cf2-9593-f5617612aad8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.544123 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d6d5e-598f-4cf2-9593-f5617612aad8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:56.546112 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7d6d5e-598f-4cf2-9593-f5617612aad8 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:56.573892 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1432/2866] 10.222.0.71 () {60 vars in 1152 bytes} [Mon Jul 18 14:28:56 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:56.578308 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.588955 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c63ec-b37e-4f8d-8448-eeeea82dfab1 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:56.654630 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019f894-8d6e-487c-8eaa-93bd60594308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:56.655319 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019f894-8d6e-487c-8eaa-93bd60594308 admin admin] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.655603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019f894-8d6e-487c-8eaa-93bd60594308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.655876 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019f894-8d6e-487c-8eaa-93bd60594308 admin admin] PATH_INFO: `/v3/users` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.656591 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019f894-8d6e-487c-8eaa-93bd60594308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:56.658561 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019f894-8d6e-487c-8eaa-93bd60594308 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:56.683314 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1435/2867] 10.222.0.71 () {60 vars in 1143 bytes} [Mon Jul 18 14:28:56 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 18 14:28:56.687468 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d6d5e-598f-4cf2-9593-f5617612aad8 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.700508 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d6d5e-598f-4cf2-9593-f5617612aad8 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:56.707542 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36c39-6aca-4a47-b263-581ae34a85f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:56.708373 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36c39-6aca-4a47-b263-581ae34a85f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.708797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36c39-6aca-4a47-b263-581ae34a85f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.709077 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f36c39-6aca-4a47-b263-581ae34a85f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.709979 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f36c39-6aca-4a47-b263-581ae34a85f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:56.711802 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f36c39-6aca-4a47-b263-581ae34a85f2 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:56.726785 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1433/2868] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 18 14:28:56.727917 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019f894-8d6e-487c-8eaa-93bd60594308 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.739331 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019f894-8d6e-487c-8eaa-93bd60594308 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:56.746838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffff9b1-c394-4217-b386-09483dadf081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:56.749647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff9b1-c394-4217-b386-09483dadf081 admin admin] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.749647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff9b1-c394-4217-b386-09483dadf081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.749647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffff9b1-c394-4217-b386-09483dadf081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.749647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffff9b1-c394-4217-b386-09483dadf081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:56.750965 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffff9b1-c394-4217-b386-09483dadf081 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:56.762789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1436/2869] 10.222.0.71 () {58 vars in 1121 bytes} [Mon Jul 18 14:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 18 14:28:56.767049 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36c39-6aca-4a47-b263-581ae34a85f2 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.778465 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f36c39-6aca-4a47-b263-581ae34a85f2 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:56.783685 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:56.784585 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] REQUEST_METHOD: `PUT` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.784867 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.785147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] PATH_INFO: `/v3/projects/9bb88a62a2924d8c8401cfaa9ed7784b/users/4de093ca793143fd98666213718fc0fe/roles/9d6a699c2b1b45629628fd317cc9c15b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.805128 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb88a62a2924d8c8401cfaa9ed7784b, user_id=4de093ca793143fd98666213718fc0fe, role_id=9d6a699c2b1b45629628fd317cc9c15b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:56.808558 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:56.832147 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1434/2870] 10.222.0.71 () {60 vars in 1482 bytes} [Mon Jul 18 14:28:56 2022] PUT /identity/v3/projects/9bb88a62a2924d8c8401cfaa9ed7784b/users/4de093ca793143fd98666213718fc0fe/roles/9d6a699c2b1b45629628fd317cc9c15b => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 18 14:28:56.843931 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224992c0-0ea3-4a1c-b5b8-76438ef0f3f7 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:56.844241 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224992c0-0ea3-4a1c-b5b8-76438ef0f3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:56.844518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224992c0-0ea3-4a1c-b5b8-76438ef0f3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:56.873313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-224992c0-0ea3-4a1c-b5b8-76438ef0f3f7 None None] MFA Rules not processed for user `4de093ca793143fd98666213718fc0fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:28:56.874600 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-224992c0-0ea3-4a1c-b5b8-76438ef0f3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:56.945833 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1437/2871] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:28:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:28:56.960010 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:28:56.962005 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:28:56.989691 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:28:57.033767 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac367fb0-b46b-4903-bd45-9c0ffaf0f70c admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:28:57.040531 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0413d-e22e-4516-a874-fb41a09a71a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '812ba8119a5a4297b409525861ec8118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:28:57.041929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0413d-e22e-4516-a874-fb41a09a71a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:28:57.042212 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0413d-e22e-4516-a874-fb41a09a71a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:28:57.042528 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0413d-e22e-4516-a874-fb41a09a71a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:28:57.052304 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0413d-e22e-4516-a874-fb41a09a71a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:28:57.053816 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b0413d-e22e-4516-a874-fb41a09a71a3 service cinder] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:28:57.063968 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1435/2872] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:28:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:29:02.708845 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224992c0-0ea3-4a1c-b5b8-76438ef0f3f7 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:29:02.710434 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-224992c0-0ea3-4a1c-b5b8-76438ef0f3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:02.730870 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:29:02.781710 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224992c0-0ea3-4a1c-b5b8-76438ef0f3f7 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:29:02.786279 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f1e67-e026-4e96-a479-0cf749b4586b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:29:02.786969 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f1e67-e026-4e96-a479-0cf749b4586b service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:02.787243 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f1e67-e026-4e96-a479-0cf749b4586b service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:02.787542 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f1e67-e026-4e96-a479-0cf749b4586b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:02.793881 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-909f1e67-e026-4e96-a479-0cf749b4586b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:02.868678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f1e67-e026-4e96-a479-0cf749b4586b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:29:02.870665 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f1e67-e026-4e96-a479-0cf749b4586b service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:29:02.883297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1438/2873] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:29:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:29:07.059287 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=17 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:29:07.646222 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0413d-e22e-4516-a874-fb41a09a71a3 service cinder] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:29:07.659439 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0413d-e22e-4516-a874-fb41a09a71a3 service cinder] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:29:07.663530 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:29:07.664189 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:07.664438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:07.664696 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:07.668012 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:07.683422 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:29:07.685617 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:29:07.704135 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1436/2874] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:29:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:29:07.897394 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aef428-7ac4-4590-9577-100c2251dee9 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:07.898238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aef428-7ac4-4590-9577-100c2251dee9 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:07.898238 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aef428-7ac4-4590-9577-100c2251dee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:07.917616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-43aef428-7ac4-4590-9577-100c2251dee9 None None] MFA Rules not processed for user `7093c86da40243a5a8e50ad0f4cd33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:29:07.918859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-43aef428-7ac4-4590-9577-100c2251dee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:07.978826 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1439/2875] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:29:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:29:11.692152 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:29:11.704942 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24e1c6-bd3e-41c1-a722-f0bd37b477c9 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:29:11.716591 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbd850-a3af-406b-969d-7fcdc2d6f0c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:29:11.717694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbd850-a3af-406b-969d-7fcdc2d6f0c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:11.717694 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbd850-a3af-406b-969d-7fcdc2d6f0c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:11.717941 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbd850-a3af-406b-969d-7fcdc2d6f0c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:11.735141 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-2cdbd850-a3af-406b-969d-7fcdc2d6f0c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:11.858709 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbd850-a3af-406b-969d-7fcdc2d6f0c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:29:11.860418 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbd850-a3af-406b-969d-7fcdc2d6f0c7 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:29:11.872002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1437/2876] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:29:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:29:16.798186 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aef428-7ac4-4590-9577-100c2251dee9 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:29:16.865163 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aef428-7ac4-4590-9577-100c2251dee9 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:29:16.871170 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:29:16.872084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:16.872366 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:16.872644 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] PATH_INFO: `/v3/users/4de093ca793143fd98666213718fc0fe` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:16.873927 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de093ca793143fd98666213718fc0fe)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:29:16.875894 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:29:16.880893 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=100 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:29:16.939580 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de093ca793143fd98666213718fc0fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:16.949678 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de093ca793143fd98666213718fc0fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:16.961916 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de093ca793143fd98666213718fc0fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:16.962800 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de093ca793143fd98666213718fc0fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:16.963528 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de093ca793143fd98666213718fc0fe'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:16.976399 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1440/2877] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:29:16 2022] DELETE /identity/v3/users/4de093ca793143fd98666213718fc0fe => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:29:16.980788 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b7c8-0662-4191-aa55-d3cc10063176 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:16.981274 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b7c8-0662-4191-aa55-d3cc10063176 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:16.981375 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b7c8-0662-4191-aa55-d3cc10063176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:16.999701 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-d0f4b7c8-0662-4191-aa55-d3cc10063176 None None] MFA Rules not processed for user `86f9463b664941f0b5c63b336b679b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:29:17.000932 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d0f4b7c8-0662-4191-aa55-d3cc10063176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:17.064789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1438/2878] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:29:17.076121 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:29:17.077700 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:17.141312 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ffee5-25f4-4e48-a9ae-06e9dd3e42b2 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:29:17.145408 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10b885-7679-4d6e-afcc-d166be3bb1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:29:17.146063 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10b885-7679-4d6e-afcc-d166be3bb1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:17.146348 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10b885-7679-4d6e-afcc-d166be3bb1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:17.146625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10b885-7679-4d6e-afcc-d166be3bb1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:17.156798 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10b885-7679-4d6e-afcc-d166be3bb1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:29:17.158297 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10b885-7679-4d6e-afcc-d166be3bb1b0 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:29:17.168530 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1441/2879] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:29:17.557119 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4b7c8-0662-4191-aa55-d3cc10063176 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:29:17.558566 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-d0f4b7c8-0662-4191-aa55-d3cc10063176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:17.571070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4b7c8-0662-4191-aa55-d3cc10063176 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:29:17.577310 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:29:17.578710 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:17.579150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:17.579436 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] PATH_INFO: `/v3/projects/9bb88a62a2924d8c8401cfaa9ed7784b` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:17.580754 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb88a62a2924d8c8401cfaa9ed7784b)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:29:17.582749 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:29:17.648346 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb88a62a2924d8c8401cfaa9ed7784b', 'request_id': 'req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:17.657203 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb88a62a2924d8c8401cfaa9ed7784b', 'request_id': 'req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:17.668089 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb88a62a2924d8c8401cfaa9ed7784b', 'request_id': 'req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:17.680250 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] The token cache is being invalidate because project 9bb88a62a2924d8c8401cfaa9ed7784b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:29:17.680543 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:17.681240 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:29:17.682962 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1439/2880] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:29:17 2022] DELETE /identity/v3/projects/9bb88a62a2924d8c8401cfaa9ed7784b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:29:27.157772 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10b885-7679-4d6e-afcc-d166be3bb1b0 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:29:27.167522 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10b885-7679-4d6e-afcc-d166be3bb1b0 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:29:27.220954 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:29:27.221635 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:29:27.221937 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:29:27.222205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:29:27.224206 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:29:27.268219 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:29:27.295098 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:29:27.296602 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:29:27.306647 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1442/2881] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:29:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:29:27.677085 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:29:37.302185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=11 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:30:13.265231 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:30:13.266768 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:30:13.279879 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678e6d1-7870-4cd7-9b9d-b602b3bbb849 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:30:13.283732 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:30:13.284380 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:30:13.284659 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:30:13.284938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:30:13.286897 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:30:13.297972 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:30:13.299438 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:30:13.308811 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1440/2882] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:30:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:30:23.304588 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=6 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:31:27.033930 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:31:27.043853 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c3208-cefb-4cc6-95b0-7ce8a6fcda1f service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:31:27.048115 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:31:27.049695 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:31:27.050001 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:31:27.050282 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:31:27.052196 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:27.063155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:31:27.064610 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:31:27.074212 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1443/2883] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:31:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:31:27.180623 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:31:27.190453 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88dab10-3213-4549-af5b-6b13bd822823 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:31:27.195009 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:31:27.195677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:31:27.195791 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:31:27.196064 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:31:27.199930 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:27.277364 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:31:27.278888 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:31:27.290849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1441/2884] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:31:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:31:27.366256 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:31:27.367944 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:27.444270 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53749323-676c-421d-9a53-6ee86822c445 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:31:27.450084 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:31:27.450758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:31:27.451043 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:31:27.451323 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:31:27.453280 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:27.466584 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:31:27.468165 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:31:27.479151 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1444/2885] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:31:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:31:29.088473 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:31:29.090173 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:29.106270 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc04da-e498-4063-bcfe-00e13c6ba46c service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:31:29.113264 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59e363-21ff-4cd5-a4f4-85f25f92241a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:31:29.114225 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59e363-21ff-4cd5-a4f4-85f25f92241a service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:31:29.114797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59e363-21ff-4cd5-a4f4-85f25f92241a service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:31:29.115075 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59e363-21ff-4cd5-a4f4-85f25f92241a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:31:29.117365 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0d59e363-21ff-4cd5-a4f4-85f25f92241a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:29.132389 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59e363-21ff-4cd5-a4f4-85f25f92241a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:31:29.135007 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59e363-21ff-4cd5-a4f4-85f25f92241a service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:31:29.149849 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1442/2886] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:31:29.265107 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:31:29.266814 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:29.288625 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbe399-7670-41fb-bedd-c238141e6ece service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:31:29.296956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:31:29.297640 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:31:29.298059 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:31:29.298551 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:31:29.300577 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:29.316305 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:31:29.319053 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:31:29.334574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1445/2887] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:31:31.247331 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74111-d81f-4299-8c15-c6e47a5c8ca9 None None] REQUEST_METHOD: `POST` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:31:31.247985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74111-d81f-4299-8c15-c6e47a5c8ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:31:31.247985 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c74111-d81f-4299-8c15-c6e47a5c8ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:31:31.267848 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.auth.core [None req-92c74111-d81f-4299-8c15-c6e47a5c8ca9 None None] MFA Rules not processed for user `7093c86da40243a5a8e50ad0f4cd33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80214) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:31:31.269082 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-92c74111-d81f-4299-8c15-c6e47a5c8ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:31.333321 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1443/2888] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:31:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:31:31.678009 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:31:31.689934 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b69889-652e-4026-98e4-bb7cc231fca8 service nova] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:31:31.694834 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:31:31.695488 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:31:31.695774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:31:31.696025 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:31:31.698320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:31.709410 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:31:31.710905 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:31:31.719527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1446/2889] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:31:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:31:34.743151 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c74111-d81f-4299-8c15-c6e47a5c8ca9 None None] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:31:34.761131 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c74111-d81f-4299-8c15-c6e47a5c8ca9 None None] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:31:34.783485 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fffb91636744fc39886bdf568317ab5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:31:34.784170 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:31:34.784443 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:31:34.784810 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:31:34.786622 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:31:34.804667 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:31:34.807590 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:31:34.820362 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1444/2890] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:31:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:31:41.715573 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=50 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:31:44.815150 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=71 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:33:00.804332 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:00.825341 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643b493-80dd-4681-9fe6-179ec32cf1d4 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:00.831234 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:00.831938 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:00.832228 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:00.832535 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:00.834456 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:00.847953 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:00.849687 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:00.860902 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1447/2891] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:33:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:33:05.206054 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:05.207949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:05.223982 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d36bae-071d-45ac-b19f-a75ad86a3a88 service nova] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:05.229833 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:05.230509 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:05.230800 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:05.231104 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:05.232996 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:05.246271 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:05.247891 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:05.261414 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1445/2892] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:33:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:33:08.655722 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:08.667761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a63d66d-51cc-4ea7-b4fc-11db51e8833a service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:08.674240 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:08.674932 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:08.675205 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:08.675616 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:08.677763 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:08.691842 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:08.693536 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:08.707371 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1448/2893] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:33:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:33:08.728086 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:08.729677 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:08.813257 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd9fbd-21cc-4052-9477-0d369fc490af service placement] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:08.818093 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:08.818981 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:08.819268 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:08.819550 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] PATH_INFO: `/v3/users/0b41303f85c7455087c8d25658f1a548` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:08.821076 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b41303f85c7455087c8d25658f1a548)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:08.823006 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:08.876074 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b41303f85c7455087c8d25658f1a548'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:08.885549 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b41303f85c7455087c8d25658f1a548'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:08.903577 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b41303f85c7455087c8d25658f1a548'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:08.904774 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b41303f85c7455087c8d25658f1a548'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:08.905921 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b41303f85c7455087c8d25658f1a548'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:08.923161 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1446/2894] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:33:08 2022] DELETE /identity/v3/users/0b41303f85c7455087c8d25658f1a548 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:33:08.933770 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:08.935626 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:08.948574 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773518ef-dd07-469b-99ee-305e9ab18b79 service placement] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:09.012313 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:09.013037 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:09.013412 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:09.013696 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:09.015977 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:09.090658 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:09.092111 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:09.101962 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1449/2895] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:33:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:33:09.287317 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:09.288948 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:09.304165 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9acfa-2eb4-402d-8a85-df13e7e3bac5 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:09.309529 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:09.310515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:09.310797 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:09.311078 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] PATH_INFO: `/v3/projects/bfe924f59fd34f58b443e1c324dc581c` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:09.312312 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe924f59fd34f58b443e1c324dc581c)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:09.314481 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:09.365539 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe924f59fd34f58b443e1c324dc581c', 'request_id': 'req-5580f691-be0c-45b7-9e27-578a38d2f4c9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.372515 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe924f59fd34f58b443e1c324dc581c', 'request_id': 'req-5580f691-be0c-45b7-9e27-578a38d2f4c9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.382100 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe924f59fd34f58b443e1c324dc581c', 'request_id': 'req-5580f691-be0c-45b7-9e27-578a38d2f4c9'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.383394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:33:09.395769 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] The token cache is being invalidate because project bfe924f59fd34f58b443e1c324dc581c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:33:09.396055 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.396722 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.398381 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1447/2896] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:33:09 2022] DELETE /identity/v3/projects/bfe924f59fd34f58b443e1c324dc581c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:33:09.404189 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:09.414444 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9c7c9-cb29-4dcd-82e6-e24f12ef1f76 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:09.477436 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:09.478260 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:09.478589 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:09.478956 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] PATH_INFO: `/v3/users/60effcfa14064f16aa7f98e286da3e42` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:09.480248 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60effcfa14064f16aa7f98e286da3e42)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:09.482239 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:09.512963 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:33:09.542235 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60effcfa14064f16aa7f98e286da3e42'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.552691 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60effcfa14064f16aa7f98e286da3e42'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.567140 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60effcfa14064f16aa7f98e286da3e42'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.567859 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60effcfa14064f16aa7f98e286da3e42'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.568501 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-0089b94e-094a-4c28-b50b-85a9a96256e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60effcfa14064f16aa7f98e286da3e42'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.580680 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1450/2897] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:33:09 2022] DELETE /identity/v3/users/60effcfa14064f16aa7f98e286da3e42 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:33:09.783505 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:09.785995 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:09.801388 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580f691-be0c-45b7-9e27-578a38d2f4c9 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:09.868048 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:09.868933 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:09.869291 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:09.869563 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] PATH_INFO: `/v3/projects/b99b488d577a4f9098769a745789f207` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:09.871041 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99b488d577a4f9098769a745789f207)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:09.873002 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:09.925918 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99b488d577a4f9098769a745789f207', 'request_id': 'req-8518ff58-c4c2-410e-b719-0ecf14beb4f0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.933820 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99b488d577a4f9098769a745789f207', 'request_id': 'req-8518ff58-c4c2-410e-b719-0ecf14beb4f0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.945070 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99b488d577a4f9098769a745789f207', 'request_id': 'req-8518ff58-c4c2-410e-b719-0ecf14beb4f0'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.958105 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] The token cache is being invalidate because project b99b488d577a4f9098769a745789f207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80214) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:33:09.958394 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.959050 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:33:09.960653 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1448/2898] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:33:09 2022] DELETE /identity/v3/projects/b99b488d577a4f9098769a745789f207 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:33:19.577320 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:DELETE=6,keystone:SELECT=19,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:33:19.954740 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:DELETE=9,keystone:SELECT=54,keystone:INSERT=1 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:33:26.193774 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ab69-31c9-441d-8291-187698083c74 None None] REQUEST_METHOD: `POST` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:26.194383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ab69-31c9-441d-8291-187698083c74 None None] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:26.194383 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ab69-31c9-441d-8291-187698083c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:26.217509 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.auth.core [None req-84b7ab69-31c9-441d-8291-187698083c74 None None] MFA Rules not processed for user `7093c86da40243a5a8e50ad0f4cd33bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80213) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 18 14:33:26.219618 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-84b7ab69-31c9-441d-8291-187698083c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:26.296886 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1451/2899] 10.222.0.71 () {58 vars in 957 bytes} [Mon Jul 18 14:33:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 18 14:33:26.668789 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:33:26.670498 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:26.737883 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518ff58-c4c2-410e-b719-0ecf14beb4f0 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:33:26.743025 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:33:26.744069 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:33:26.744237 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:33:26.744493 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:33:26.747149 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:33:26.762343 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:33:26.764374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:33:26.773399 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1449/2900] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:33:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:33:36.285236 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=25 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:33:36.769374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=26 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:35:10.936761 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7ab69-31c9-441d-8291-187698083c74 None None] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:10.938576 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-84b7ab69-31c9-441d-8291-187698083c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:10.952543 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7ab69-31c9-441d-8291-187698083c74 None None] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:10.957013 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:10.957706 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:10.957991 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:10.958247 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:10.960199 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:10.972166 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:10.973608 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:10.987481 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1452/2901] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:35:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:35:12.130893 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:12.142319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5ec97-28a7-45d9-9c9e-f297617dd5f4 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:12.146358 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:12.147027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:12.147316 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:12.147555 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:12.149799 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:12.252911 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:12.254413 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:12.266361 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1450/2902] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:35:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:35:13.217285 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:13.227123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a425c23-5dec-4835-810b-d72410f3ec67 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:13.232396 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:13.233055 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:13.233333 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:13.233603 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:13.236368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:13.247968 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:13.249425 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:13.258214 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1453/2903] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:35:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:35:13.456232 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:13.465986 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bef01c-68cb-4d03-a623-60f8b8de47c6 service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:13.470226 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:13.470889 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:13.471146 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:13.471425 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:13.473421 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:13.489762 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:13.490593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:13.502831 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1451/2904] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:35:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:35:23.254143 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=12 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:35:23.497506 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=40 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:35:24.762673 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:24.772090 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1611ba-8c26-4827-adff-6bf50ce349c9 service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:24.775838 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f1fe084c3164e50bf5d904350925f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:24.776503 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] REQUEST_METHOD: `GET` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:24.776789 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:24.777065 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:24.779152 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:24.858496 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:24.859960 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:24.870449 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1454/2905] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:35:31.807428 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:31.809363 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:31.828817 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450cb0b-786f-499c-8160-e37d3234724b service neutron] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:31.836446 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:31.841218 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:31.841520 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:31.842624 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] PATH_INFO: `/v3/users/7093c86da40243a5a8e50ad0f4cd33bb` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:31.844925 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7093c86da40243a5a8e50ad0f4cd33bb)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:31.846964 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:31.918593 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7093c86da40243a5a8e50ad0f4cd33bb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:31.931949 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7093c86da40243a5a8e50ad0f4cd33bb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:31.948101 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7093c86da40243a5a8e50ad0f4cd33bb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:31.948938 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7093c86da40243a5a8e50ad0f4cd33bb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:31.949628 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7093c86da40243a5a8e50ad0f4cd33bb'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:31.964022 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1452/2906] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:35:31 2022] DELETE /identity/v3/users/7093c86da40243a5a8e50ad0f4cd33bb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:35:32.197114 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:32.210836 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a1788d-5fd6-40f4-9c92-6e8d66a485ab service neutron] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:32.277928 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:32.278848 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:32.279155 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:32.279424 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] PATH_INFO: `/v3/projects/7a000079ba214fdf934764899139f62d` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:32.280683 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a000079ba214fdf934764899139f62d)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:32.282663 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:32.336277 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a000079ba214fdf934764899139f62d', 'request_id': 'req-a1b92aa5-05c8-4d7a-8986-df298d946bf7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.346123 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a000079ba214fdf934764899139f62d', 'request_id': 'req-a1b92aa5-05c8-4d7a-8986-df298d946bf7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.352785 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a000079ba214fdf934764899139f62d', 'request_id': 'req-a1b92aa5-05c8-4d7a-8986-df298d946bf7'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.363939 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] The token cache is being invalidate because project 7a000079ba214fdf934764899139f62d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:35:32.364221 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.364918 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.366758 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1455/2907] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:35:32 2022] DELETE /identity/v3/projects/7a000079ba214fdf934764899139f62d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:35:32.370900 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:32.372374 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:32.386778 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38099977-38b4-4f80-8908-749e21ff4d4f admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:32.447313 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:32.448108 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:32.448395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:32.448652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] PATH_INFO: `/v3/users/f24bc48b705f4f4dbb3ad78e29dead11` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:32.450427 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f24bc48b705f4f4dbb3ad78e29dead11)` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:32.453395 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:32.509434 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f24bc48b705f4f4dbb3ad78e29dead11'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.519027 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f24bc48b705f4f4dbb3ad78e29dead11'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.533545 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f24bc48b705f4f4dbb3ad78e29dead11'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.534445 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f24bc48b705f4f4dbb3ad78e29dead11'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.535139 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.notifications [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f24bc48b705f4f4dbb3ad78e29dead11'} {{(pid=80214) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.537652 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:35:32.555411 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1453/2908] 10.222.0.71 () {58 vars in 1223 bytes} [Mon Jul 18 14:35:32 2022] DELETE /identity/v3/users/f24bc48b705f4f4dbb3ad78e29dead11 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:35:32.747368 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] Authenticating user token {{(pid=80213) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:35:32.748891 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.fernet_utils [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80213) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:35:32.763998 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b92aa5-05c8-4d7a-8986-df298d946bf7 admin admin] Validating token access rules against request {{(pid=80213) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:35:32.788378 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:35:32.831078 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86f9463b664941f0b5c63b336b679b58', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1db40e87aaf497fb0528cd53913ada9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80213) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:35:32.832149 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:35:32.832518 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:35:32.832860 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] PATH_INFO: `/v3/projects/766435eed1ca42d697b0c97032bc40d4` {{(pid=80213) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:35:32.834185 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=766435eed1ca42d697b0c97032bc40d4)` {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:35:32.836231 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] RBAC: Authorization granted {{(pid=80213) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:35:32.887806 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '766435eed1ca42d697b0c97032bc40d4', 'request_id': 'req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.897244 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '766435eed1ca42d697b0c97032bc40d4', 'request_id': 'req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.905302 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '766435eed1ca42d697b0c97032bc40d4', 'request_id': 'req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056'} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.917188 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] The token cache is being invalidate because project 766435eed1ca42d697b0c97032bc40d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80213) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 18 14:35:32.917479 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.918374 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG keystone.notifications [None req-40e7f8bf-afb1-4ea8-9b8c-23fe59155056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80213) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 18 14:35:32.920144 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: [pid: 80213|app: 0|req: 1456/2909] 10.222.0.71 () {58 vars in 1232 bytes} [Mon Jul 18 14:35:32 2022] DELETE /identity/v3/projects/766435eed1ca42d697b0c97032bc40d4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 18 14:35:42.550929 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=4 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:35:42.913527 dv-cinder-797608-58-iscsi devstack@keystone.service[80213]: DEBUG dbcounter [-] [80213] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80213) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 18 14:38:24.525372 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] Authenticating user token {{(pid=80214) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 18 14:38:24.527366 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:38:24.687794 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c75fc-442c-4ec5-9b12-997bee865890 admin admin] Validating token access rules against request {{(pid=80214) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 18 14:38:24.765023 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2ceaf-4f2a-40b9-9c2a-5ddf195d9695 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8895e6cf7a1e445c825f391309766d76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83aa8da005274151aa6d73a3e6fbaa7c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80214) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 18 14:38:24.765757 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ceaf-4f2a-40b9-9c2a-5ddf195d9695 service placement] REQUEST_METHOD: `GET` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 18 14:38:24.766043 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ceaf-4f2a-40b9-9c2a-5ddf195d9695 service placement] SCRIPT_NAME: `/identity` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 18 14:38:24.766319 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2ceaf-4f2a-40b9-9c2a-5ddf195d9695 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80214) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 18 14:38:24.768242 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.fernet_utils [None req-ead2ceaf-4f2a-40b9-9c2a-5ddf195d9695 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80214) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 18 14:38:24.782137 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2ceaf-4f2a-40b9-9c2a-5ddf195d9695 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 18 14:38:24.784676 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead2ceaf-4f2a-40b9-9c2a-5ddf195d9695 service placement] RBAC: Authorization granted {{(pid=80214) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 18 14:38:24.798666 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: [pid: 80214|app: 0|req: 1454/2910] 10.222.0.71 () {60 vars in 1365 bytes} [Mon Jul 18 14:38:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 18 14:38:34.792234 dv-cinder-797608-58-iscsi devstack@keystone.service[80214]: DEBUG dbcounter [-] [80214] Writing DB stats keystone:SELECT=89 {{(pid=80214) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}